Law of Large Numbers

Sample averages converge to the true mean: X_n → .

The idea

Theorem (Law of Large Numbers).
Let $X_1, X_2, \ldots$ be independent draws from the same distribution, with finite mean $\mathbb{E}[X_i] = \mu$, and let $\bar X_n = \frac{1}{n}\sum_{i=1}^{n} X_i$ be the running average. Then for every tolerance $\epsilon > 0$, the probability $\mathbb{P}(|\bar X_n - \mu| \geq \epsilon)$ tends to $0$ as $n$ grows (the weak law), and with probability $1$ the sequence $\bar X_1, \bar X_2, \ldots$ converges to $\mu$ (the strong law).

The two versions make "the average converges to $\mu

quot; precise in different ways: the weak law bounds the chance of a large error at each fixed $n$, and the strong law speaks about the whole sequence at once.

This is why we can trust an average of many measurements where we cannot trust a single one, and why we can estimate an unknown expectation by drawing samples and averaging them.

The law does not say that a run of heads makes tails more likely next: no future draw corrects a past imbalance. An excess of ten heads is never repaid; it is divided by an ever larger $n$ until it stops mattering.

Ways to work on it

Walkthrough. Prove the weak LLN via Chebyshev and pin down the role of each assumption.
Practice. Numerical Chebyshev bound on sample-mean deviation.
Hardest. Edge cases — when averaging fails, and what the stronger forms of the law say.

Not sure where to start? Take the ten-question placement test.