Second-order Taylor expansion

Statement

Let $U \subseteq \mathbb{R}^n$ be open, let $f \colon U \to \mathbb{R}$ be of class C^2 on $U$, and let $a \in U$. Let $\nabla f(a)$ be the gradient of $f$ at $a$, and let $H_f(a)$ be the Hessian matrix of $f$ at $a$. Then $$f(a + h) = f(a) + \nabla f(a) \cdot h + \tfrac{1}{2} \, h^{\mathsf{T}} H_f(a) \, h + R(h), \qquad \lim_{h \to 0} \frac{R(h)}{\|h\|^2} = 0.$$

Remarks

This is the case $k = 2$ of Taylor's theorem in several variables, written with matrices. The quadratic term collects the terms with $|\alpha| = 2$. For $n = 2$ and $h = (u, v)$, the quadratic term is $\tfrac{1}{2} \left( f_{xx}(a) u^2 + 2 f_{xy}(a) u v + f_{yy}(a) v^2 \right)$. At a critical point the linear term is zero, so the sign of $h^{\mathsf{T}} H_f(a) h$ controls the behaviour of $f$ near $a$. This is the basis of the second derivative test.

Explanations

L1 human

A function of class $C^2$ is close to a quadratic polynomial near each point. The gradient gives the linear part and the Hessian gives the quadratic part. The error goes to zero faster than $\|h\|^2$.