Chain Rule as Composition

The derivative of a composition is the matrix product of the derivative maps.

The idea

Theorem (Chain rule).

Let $f \colon \mathbb{R}^{m} \to \mathbb{R}^{n}$ be differentiable at $a$, and let $g \colon \mathbb{R}^{n} \to \mathbb{R}^{p}$ be differentiable at $f(a)$. Then $g \circ f$ is differentiable at $a$, and $D(g \circ f)(a) = Dg(f(a))\,Df(a).$

Here $Df(a)$ is the derivative in its multivariable sense: the linear map, recorded as an $n \times m$ matrix, for which $f(a + h) = f(a) + Df(a)h + r(h), \qquad \frac{|r(h)|}{|h|} \to 0 \text{ as } h \to 0.$ To say $f$ is differentiable at $a$ is to say that such a linear map exists.

The theorem says that composing two maps composes their linear approximations. Near $a$ the map $f$ behaves like the linear map $Df(a)$; near the image point $f(a)$ the map $g$ behaves like $Dg(f(a))$; so near $a$ the composition behaves like the two linear maps applied one after the other, and applying linear maps in succession is matrix multiplication.

The outer derivative is evaluated at $f(a)$, not at $a$, because $f(a)$ is the point at which $g$ receives its input. The order of the factors is also fixed: with sizes $p \times n$ and $n \times m$, the product is defined only as written. Multiplying it out entry by entry recovers the familiar chain rules written as sums of products of partial derivatives.

Ways to work on it

• Walkthrough. The chain rule as composition of linear maps — a matrix product.
• Practice. Read off an entry of the chain-rule matrix product.
• Hardest. The chain rule on the unit circle and its geometric meaning.

Not sure where to start? Take the ten-question placement test.