Bayes' Theorem

P(A | B) = P(B | A) P(A)/P(B) — flip conditionals.

The idea

Theorem (Bayes' theorem).

For events $A$ and $B$ with $\mathbb{P}(B) > 0$, $\mathbb{P}(A \mid B) = \frac{\mathbb{P}(B \mid A)\,\mathbb{P}(A)}{\mathbb{P}(B)}.$

The theorem converts one direction of conditioning into the other. The two directions are different numbers, and it is often one of them we can measure and the other we want. A manufacturer can measure $\mathbb{P}(+ \mid D)$, the probability that a person with disease $D$ tests positive, by testing people known to have it; a patient holding a positive result wants $\mathbb{P}(D \mid +)$.

The denominator $\mathbb{P}(B)$ collects every way $B$ can occur.

Lemma (Law of total probability).

If exactly one of the events $A_{1}, \ldots, A_{n}$ must occur, then $\mathbb{P}(B) = \sum_{i} \mathbb{P}(B \mid A_{i})\,\mathbb{P}(A_{i}).$

In the figure, the positives come from two columns: the narrow column $D$, most of which tests positive, and the wide column $\neg D$, a small share of which does. When $D$ is rare, the second region is the larger, so most positives are false alarms even though the test is accurate. The denominator is what carries this fact into the theorem.

Ways to work on it

• Walkthrough. The classic disease-testing problem.
• Proof. See why — derive Bayes from the definition of conditional probability plus total probability.
• Practice. Two-step Bayes on a friendlier prevalence.
• Hardest. Find the probability of disease from a positive test when the disease is rare.

Not sure where to start? Take the ten-question placement test.