Orthogonal Projections

Closest point on a subspace, and the projection matrix.

The idea

The orthogonal projection of a vector $b$ onto a line through the origin is the point $p$ of that line closest to $b$.

Let $a$ span the line, so every point of it is a multiple $\hat{x}\,a$, and write $e = b - \hat{x}\,a$ for the error this choice leaves. If $e$ had a component along $a$, moving the point in that direction would shorten the error, so $e$ is as short as possible exactly when it is perpendicular to $a$. Writing that condition as $a^{T}(b - \hat{x}\,a) = 0$ and solving for $\hat{x}$ gives $\hat{x} = \dfrac{a^{T} b}{a^{T} a}$, so the closest point is $p = \frac{a^{T} b}{a^{T} a}\,a = \frac{a a^{T}}{a^{T} a}\,b.$ The second form regroups the first so that $b$ stands alone on the right. It exhibits the projection as one fixed matrix, $P = \dfrac{a a^{T}}{a^{T} a}$, applied to $b$ — a matrix that projects every vector onto the line.

Ways to work on it

• Walkthrough. Project onto a line, the error condition, and the projection matrix.
• Practice. Project a vector onto a line and find the scalar.
• Hardest. Build the projection matrix and use P^2 = P.

Not sure where to start? Take the ten-question placement test.