Proof that the orthogonal projection is the closest point

Proof

We write $v - w$ as a sum of two orthogonal vectors and apply the Pythagorean theorem.

1. Let $V$ be an inner product space over $\R$ or $\C$, with inner product $\ip{\cdot}{\cdot}$ and induced norm $\norm{\cdot}$. Let $W$ be a finite-dimensional subspace, let $v \in V$ and let $w \in W$.
2. By the definition of the orthogonal projection, $v = P_W(v) + w'$ with $P_W(v) \in W$ and $w' = v - P_W(v) \in W^\perp$. Here $W^\perp$ is the orthogonal complement.
3. Set $a = v - P_W(v)$ and $b = P_W(v) - w$. Then $v - w = a + b$.
4. Since $W$ is a subspace and $P_W(v), w \in W$, we have $b \in W$. Since $a \in W^\perp$, we get $\ip{a}{b} = 0$. So $a$ and $b$ are orthogonal.
5. By the Pythagorean theorem, $\norm{v - w}^2 = \norm{a}^2 + \norm{b}^2$.
6. Since $\norm{b}^2 \ge 0$, this gives $\norm{v - w}^2 \ge \norm{a}^2 = \norm{v - P_W(v)}^2$.
7. Both norms are nonnegative, and the square root is increasing on $[0, \infty)$. So $\norm{v - P_W(v)} \le \norm{v - w}$.
8. Suppose equality holds. Then the squares are equal, and step 5 gives $\norm{b}^2 = 0$.
9. So $\ip{b}{b} = 0$, and positivity of the inner product gives $b = 0$. That is, $w = P_W(v)$.
10. Conversely, if $w = P_W(v)$, then both sides equal $\norm{v - P_W(v)}$. So equality holds only for $w = P_W(v)$.