Proof that the gradient points in the direction of steepest ascent

Proof

We write the directional derivative as a dot product with the gradient and compare it with its largest possible value.

1. Let $U$, $f$ and $a$ be as in the statement. Put $g = \nabla f(a)$, so $g \ne 0$. Let $u \in \mathbb{R}^n$ with $\|u\| = 1$, and put $w = g / \|g\|$, a unit vector.
2. Since the derivative gives the directional derivatives, the directional derivative $D_u f(a)$ exists and equals $Df(a)(u)$.
3. By the definition of the gradient, $Df(a)(u) = g \cdot u$. So $D_u f(a) = g \cdot u$.
4. By the Cauchy–Schwarz inequality in Euclidean space, $|g \cdot u| \le \|g\| \, \|u\| = \|g\|$. So $-\|g\| \le D_u f(a) \le \|g\|$.
5. Expanding the dot product, $\|u - w\|^2 = \|u\|^2 - 2 \, u \cdot w + \|w\|^2 = 2 - 2 \, \dfrac{g \cdot u}{\|g\|}$.
6. In the same way, $\|u + w\|^2 = 2 + 2 \, \dfrac{g \cdot u}{\|g\|}$.
7. By steps 3 and 5, $D_u f(a) = \|g\|$ holds if and only if $\|u - w\| = 0$, that is, if and only if $u = g / \|g\|$.
8. By steps 3 and 6, $D_u f(a) = -\|g\|$ holds if and only if $\|u + w\| = 0$, that is, if and only if $u = -g / \|g\|$.
9. So the upper bound is attained exactly at $u = \nabla f(a) / \|\nabla f(a)\|$, and the lower bound exactly at $u = -\nabla f(a) / \|\nabla f(a)\|$.