Convergence to equilibrium of ergodic Markov chains

Remarks

The classical proof is by coupling. Run a second independent copy $(Y_n)$ started from $\pi$; aperiodicity makes the pair $(X_n, Y_n)$ an irreducible chain with a stationary distribution. Hence the pair is recurrent and hits the diagonal at a finite time $\tau$, and $\sum_j \abs{P(X_n = j) - \pi_j} \le 2 P(\tau > n) \to 0$. Example: on $S = \set{1, 2}$ let $p_{12} = a$ and $p_{21} = b$ with $0 < a, b \le 1$ and $a + b < 2$. Then $\pi = (b, a)/(a + b)$ and $P(X_n = 1) - \pi_1 = (1 - a - b)^n \left(P(X_0 = 1) - \pi_1\right)$, which tends to $0$. Aperiodicity is needed: for $a = b = 1$ the chain alternates, and $P(X_n = 1)$ oscillates between $0$ and $1$ when $X_0 = 1$. Irreducibility is needed: with two absorbing states the limit depends on the starting state.

Going deeper

For a student

The theorem justifies Markov chain Monte Carlo: running an ergodic chain produces samples whose law approaches $\pi$. The speed of convergence is measured by the mixing time, and for finite chains it is geometric, governed by the second largest eigenvalue modulus of $P$. Without aperiodicity, the averages $\frac1n \sum_{k=1}^{n} P(X_k = j)$ still converge to $\pi_j$. For a finite state space, condition 3 follows from condition 1.