Courant–Fischer min-max theorem

Remarks

Let $u_1, \dots, u_n$ be an orthonormal basis of eigenvectors with $A u_i = \lambda_i u_i$. On the span of $u_1, \dots, u_k$, the Rayleigh quotient is at least $\lambda_k$, so the max-min is at least $\lambda_k$. Every subspace $U$ of dimension $k$ meets the span of $u_k, \dots, u_n$ in a nonzero vector, by counting dimensions, and there $R_A \le \lambda_k$. So the max-min is exactly $\lambda_k$; the min-max form is proved in the same way.

For $k = 1$ it gives $\lambda_1 = \max_{x \ne 0} R_A(x)$, and for $k = n$ it gives $\lambda_n = \min_{x \ne 0} R_A(x)$. For $A = \operatorname{diag}(3, 1)$, $R_A(x) = (3x_1^2 + x_2^2)/(x_1^2 + x_2^2)$ for real $x$, which takes all values in $[1, 3]$.

The Hermitian hypothesis is needed: for a non-Hermitian matrix the eigenvalues need not be real, and $R_A$ need not be real either.

Going deeper

For a student

The theorem turns questions about eigenvalues into questions about subspaces. It gives the Cauchy interlacing theorem and Weyl's inequality in a few lines. The real symmetric case is the same, with $\R^n$ and $x^T A x$. A version holds for compact self-adjoint operators on Hilbert spaces.