Karhunen–Loève theorem

Remarks

The proof rests on Mercer's theorem: $K$ is continuous, symmetric and positive semidefinite, so $K(s, t) = \sum_k \lambda_k e_k(s) e_k(t)$ with uniform convergence on $[a, b]^2$. Fubini's theorem gives the moments of the $Z_k$ and $E[X_t Z_k] = \lambda_k e_k(t)$. Expanding the square then gives $$E\left[\left(X_t - \sum_{k \le n} Z_k e_k(t)\right)^2\right] = K(t, t) - \sum_{k \le n} \lambda_k e_k(t)^2,$$ which tends to $0$ uniformly in $t$ by Mercer's theorem.

Example: for the standard Wiener process on $[0, 1]$, $K(s, t) = \min(s, t)$. Solving the eigenvalue equation gives $e_k(t) = \sqrt{2} \sin\left((k - \tfrac{1}{2})\pi t\right)$ and $\lambda_k = \dfrac{1}{(k - \frac{1}{2})^2 \pi^2}$ for $k \ge 1$. So $W_t = \sum_{k \ge 1} \sqrt{2} \, \xi_k \dfrac{\sin\left((k - \frac{1}{2})\pi t\right)}{(k - \frac{1}{2})\pi}$ with i.i.d. standard normal $\xi_k$.

Going deeper

For a student

The expansion is the continuous analogue of principal component analysis, which diagonalises a covariance matrix. Truncating it after $n$ terms gives the best approximation by $n$ terms for the mean square error integrated over $[a, b]$, among all expansions in an orthonormal basis of $L^2[a, b]$. It is used to simulate Gaussian processes, to reduce dimension in signal processing and uncertainty quantification, and to compute laws of quadratic functionals such as $\int_0^1 W_t^2 \, dt = \sum_k \lambda_k \xi_k^2$.