Convexity in Several Variables

The chord definition survives the jump to vectors; the tests become the gradient inequality and a positive semidefinite Hessian.

The idea

Convexity for a function of several variables keeps the definition from Convex Functions and replaces the test. On a convex domain, $f$ is convex when $f(\theta x + (1 - \theta) y) \le \theta f(x) + (1 - \theta) f(y)$ for all $x, y$ in the domain and all $0 \le \theta \le 1$ — the same chord condition as before, applied one segment at a time.

The test $f'' \ge 0$ must change, because a function of several variables has no single second derivative. It has a matrix of them, the Hessian $\nabla^{2} f(x)$, whose $(i,j)$ entry is $\partial^{2} f / \partial x_{i} \partial x_{j}$, and $f$ is convex exactly when the Hessian is positive semidefinite at every point, meaning $v^{T} \nabla^{2} f(x) v \ge 0$ for every vector $v$. That quantity is the second derivative of $f$ along the line through $x$ in the direction $v$, so the condition says that $f$ curves upward in every direction.

For a symmetric $2 \times 2$ matrix we can carry out the check by hand. When $a > 0$, completing the square gives $v^{T} \begin{pmatrix} a & b \\ b & c \end{pmatrix} v = a\left(v_{1} + \frac{b}{a} v_{2}\right)^{2} + \frac{ac - b^{2}}{a} v_{2}^{2},$ and choosing $v$ to isolate either square shows the expression is nonnegative for every $v$ exactly when $a \ge 0$, $c \ge 0$, and $ac - b^{2} \ge 0$.

Ways to work on it

• Walkthrough. Test a function of several variables for convexity using its gradient and Hessian.
• Practice. Classify a random quadratic as convex, concave, or neither from its Hessian.
• Hardest. Find the range of a parameter that keeps a quadratic convex.

Not sure where to start? Take the ten-question placement test.