Operator Norm

|A|_ op = _|x|=1 |Ax| = _ (A).

The idea

The operator norm measures how strongly a matrix stretches vectors. A matrix stretches different directions by different amounts, so no single factor describes it exactly; the operator norm records the worst case. Fix the input length at $1$ and take the largest output length: $\|A\|_{\mathrm{op}} = \sup_{\|x\| = 1} \|Ax\|.$ Geometrically, $A$ carries the unit ball to an ellipsoid, and $\|A\|_{\mathrm{op}}$ is the length of that ellipsoid's longest semi-axis. The figure shows the planar case: $A$ carries the unit circle's perpendicular directions $v_{1}, v_{2}$ to the ellipse's perpendicular semi-axes $\sigma_{1} u_{1}$ and $\sigma_{2} u_{2}$, and the longer stretch $\sigma_{1}$ — the largest singular value of $A$ — is exactly $\|A\|_{\mathrm{op}}$.

Two consequences follow directly. Rescaling the input shows that $\|Ax\| \leq \|A\|_{\mathrm{op}} \|x\|$ for every $x$, not only unit vectors, so the norm bounds each output by its input. Applying that bound twice to $A(Bx)$ gives $\|AB\|_{\mathrm{op}} \leq \|A\|_{\mathrm{op}} \|B\|_{\mathrm{op}},$ the property called sub-multiplicativity.

Ways to work on it

• Walkthrough. The operator norm as the largest stretch of a unit vector, computed for diagonal and symmetric matrices.
• Practice. Compute |A|_ op for a small diagonal or symmetric matrix.
• Hardest. Sub-multiplicativity and the operator norm of the identity.

Not sure where to start? Take the ten-question placement test.