Pontryagin's maximum principle

Remarks

The proof compares the optimal control with needle variations, which replace $u^*$ by a constant value $v \in U$ on a short interval. The first-order effects of these variations on the endpoint and the cost form a convex cone. Optimality means that this cone does not meet a certain direction. A separating hyperplane gives $(\lambda_0, p(T))$, and $p$ is then transported backward by the adjoint equation of condition 1. Example: minimize $\int_0^1 \frac{1}{2} u^2 \, dt$ for $x' = u$, $x(0) = 0$, $x(1) = 1$, $U = \R$. Here $H = p u - \lambda_0 u^2 / 2$ and $p' = 0$, so $p$ is constant; if $\lambda_0 = 0$, condition 2 forces $p = 0$, which is excluded, so $\lambda_0 = 1$ and $u = p$ is constant, which gives $u \equiv 1$ and $x(t) = t$. The theorem gives necessary conditions only: an extremal that satisfies them need not be optimal. The case $\lambda_0 = 0$, called abnormal, can occur, and then the conditions do not involve the cost at all.

Going deeper

For a student

The maximum principle extends the Euler–Lagrange equations of the calculus of variations to controls with values in a constraint set $U$, such as $[-1, 1]$. When $H$ is linear in $u$ and $U$ is a box, condition 2 forces $u$ to take extreme values, which gives bang–bang controls except on singular arcs. The state equation $x' = \partial H / \partial p$ and the adjoint equation form a Hamiltonian system with boundary conditions at both ends, usually solved by shooting methods. Variants with free final time add the condition that the constant in condition 3 is zero. Variants with a free endpoint and no terminal cost add the transversality condition $p(T) = 0$.