De Finetti's theorem

Remarks

In words, the sequence is a mixture of i.i.d. sequences: first draw a law $m$ from $\nu$, then draw $X_1, X_2, \dots$ i.i.d. with law $m$. For $S = \set{0, 1}$ the theorem says that there is a unique probability measure $\mu$ on $[0, 1]$ with $$P(X_1 = x_1, \dots, X_n = x_n) = \int_0^1 p^k (1 - p)^{n - k} \, \mu(dp), \qquad k = x_1 + \dots + x_n,$$ so the sequence is a mixture of Bernoulli sequences. In this case the random parameter is the almost sure limit of the frequencies $(X_1 + \dots + X_n)/n$, and its law is $\mu$. One proof shows that, given the σ-algebra of events invariant under finite permutations, the variables are conditionally i.i.d.; a reverse martingale argument gives the limit of the empirical measures. The theorem fails for finite sequences: drawing both balls without replacement from an urn with one red and one blue ball gives an exchangeable pair $(X_1, X_2)$ with $P(X_1 = X_2 = 1) = 0$ and $P(X_1 = X_2 = 0) = 0$. A mixture would need $\int p^2 \, \mu(dp) = 0$ and $\int (1 - p)^2 \, \mu(dp) = 0$, which is impossible. In the $\set{0,1}$ case, $\operatorname{Cov}(X_1, X_2) = \int p^2 \, d\mu - \left(\int p \, d\mu\right)^2 \ge 0$, so infinite exchangeable sequences of this kind are never negatively correlated.

Going deeper

For a student

The theorem is the main justification for the Bayesian model "parameter with a prior, data i.i.d. given the parameter": the measure $\nu$ plays the role of the prior. The Pólya urn model gives an example: its colour sequence is exchangeable, and the mixing measure is a beta distribution. For general exchangeable sequences, the Dirichlet process is a standard choice of $\nu$ in nonparametric Bayesian statistics. The Hewitt–Savage zero–one law is a related result: for an i.i.d. sequence, the exchangeable σ-algebra is trivial, which matches $\nu$ being a point mass.