Stationary Distributions

Where a Markov chain settles: the distribution one step leaves unchanged.

The idea

A stationary distribution of a Markov chain is a distribution over the states that one step of the chain leaves unchanged: draw today's state from $\pi$, take one transition, and tomorrow's state is again distributed as $\pi$. In symbols,

$\pi = \pi P,$

where $P$ is the matrix of transition probabilities and $\pi P$ is the distribution one step later. A stationary distribution is a fixed point of this one-step update.

The equation concerns the distribution, not the trajectory. An individual run of the chain keeps hopping from state to state; what stays fixed is the amount of probability resting on each state.

Stationary distributions describe the long run: for a broad class of chains, the distribution of the state after $n$ steps approaches $\pi$ whatever the starting state, so $\pi_{i}$ is the long-run fraction of time the chain spends in state $i$.

To find $\pi$, solve linear equations. The relation $\pi = \pi P$ gives one equation per state, but they are not independent — because each row of $P$ sums to $1$, any one of the equations follows from the others. The normalization $\sum_{i} \pi_{i} = 1$ supplies the missing condition and pins $\pi$ down.

Ways to work on it

• Walkthrough. What it means for a chain's distribution to hold steady, and how to find the one that does.
• Practice. Solve balance plus normalization for a two-state chain.
• Hardest. Balance a three-state chain, state by state.

Not sure where to start? Take the ten-question placement test.