Hamilton–Jacobi–Bellman equation

Remarks

Its intended solution is the value function $V(t, x) = \inf_u \left( \int_t^T L(x(s), u(s)) \, ds + g(x(T)) \right)$, where the infimum is over admissible controls on $[t, T]$ and $x$ is the trajectory with $x(t) = x$. If the value function is continuously differentiable, it satisfies the equation. Under standard hypotheses, such as $U$ compact and $f$, $L$, $g$ bounded and Lipschitz, it is the unique bounded uniformly continuous viscosity solution. Example: for $x' = u$, $U = \R$, $L = u^2 / 2$ and $g(x) = x^2 / 2$, the minimum over $u$ is $-\frac{1}{2} (\partial_x V)^2$, and $V(t, x) = \dfrac{x^2}{2 (1 + T - t)}$ solves the equation. Indeed $\partial_t V = \dfrac{x^2}{2 (1 + T - t)^2}$ and $\frac{1}{2} (\partial_x V)^2 = \dfrac{x^2}{2 (1 + T - t)^2}$, and $V(T, x) = x^2 / 2$. Non-example: $W(t, x) = x^2 / 2$ satisfies the terminal condition but not the equation, since $\partial_t W - \frac{1}{2} (\partial_x W)^2 = -x^2 / 2 \ne 0$ for $x \ne 0$. For infinite-horizon problems with discount rate $\rho > 0$ the equation becomes stationary: $\rho V(x) = \min_{u \in U} \{ \nabla V(x) \cdot f(x, u) + L(x, u) \}$.