Show Question
Jensen's inequality

If $$f$$ is a convex (smile) function and $$X$$ is a random variable then:

$E[f(X)] \ge f(E[X])$
If $$f$$ is strictly convex and $$E[f(X)] = f(E[X])$$, then the random variable $$X$$ is a constant.

Intuition:

Proof:

Intuition #2

The following is taken from Andrew Ng's notes for CS229:

TODO

Example

Q1: Three squares

Three squares have average area $$\bar{A} = 100 m^2$$. The average of the lengths of their sides is $$\bar{l} = 10 m$$. What can be said about the size of the largest of the three squares?

A1:
Let $$x$$ be the length of the side of a square, and let the probability of $$x$$ be $$\frac{1}{3}, \frac{1}{3}, \frac{1}{3}$$ over the three lengths, $$l_1, l_2, l_3$$. Then the information that we have is:

* $$E[X] = 10$$
* $$E[f(X)] = 100 \text{, where } f(x) = x^2$$

$$f$$ is a strictly convex function and the equality $$E[f(X)] = f(E[X])$$ holds, so by Jensen's equality, $$x$$ must be a constant and all three lengths must be equal. So the area of the largest square (and all squares) is $$100 m^2$$.