Jensen's Inequality

Convex f: f(E[X]) ≤ E[f(X)].

The idea

Theorem (Jensen's inequality).

Let $f$ be a convex function and $X$ a random variable. Then $f(\mathbb{E}[X]) \;\leq\; \mathbb{E}[f(X)].$ If $f$ is concave the inequality reverses, and when $f$ is strictly convex equality holds if and only if $X$ is constant.

The two sides differ in the order of two operations. On the left, $X$ is averaged first and $f$ is applied once, to the average. On the right, $f$ is applied to each outcome and the resulting values are averaged. Jensen's inequality says the second quantity is never smaller.

The reason is the shape of a convex graph: the chord joining two points on the graph lies on or above the arc between them. For an $X$ taking two values with equal probability, $\mathbb{E}[f(X)]$ is the height of the chord's midpoint, and $f(\mathbb{E}[X])$ is the height of the curve directly below that midpoint, so the chord comparison is the inequality itself. The general statement extends this comparison to arbitrary outcomes and weights.

Jensen's inequality is the standard tool for moving an expectation past a nonlinear function.

Ways to work on it

• Walkthrough. Verify Jensen's inequality on a two-point random variable.
• Proof. Derive Jensen from the support line of a convex f at E[X].
• Practice. Compute f(E[X]) for a random 2-point X.
• Hardest. Compute the full Jensen gap.

Not sure where to start? Take the ten-question placement test.