---
tag: CP25WM5537
kind: definition
name: Hamilton–Jacobi–Bellman equation
msc: [49L12, 49L20]
defines: SJP9HW82VX
uses: [ZPDTVZQN5K]
sources:
  - url: https://en.wikipedia.org/wiki/Hamilton–Jacobi–Bellman_equation
---

## Definition

Let $U \subseteq \R^m$ be a nonempty set, let $T > 0$, and let $f \colon \R^n \times U \to \R^n$, $L \colon \R^n \times U \to \R$ and $g \colon \R^n \to \R$ be continuous.
Consider the [[control system|ZPDTVZQN5K]] $x' = f(x, u)$ with running cost $L$ and terminal cost $g$.
The *Hamilton–Jacobi–Bellman equation* of this problem is the partial differential equation
$$\frac{\partial V}{\partial t}(t, x) + \min_{u \in U} \left\{ \nabla_x V(t, x) \cdot f(x, u) + L(x, u) \right\} = 0, \qquad (t, x) \in (0, T) \times \R^n,$$
with terminal condition $V(T, x) = g(x)$, for an unknown function $V \colon [0, T] \times \R^n \to \R$.
When the minimum is not attained, it is replaced by the infimum.

## Remarks

Its intended solution is the value function $V(t, x) = \inf_u \left( \int_t^T L(x(s), u(s)) \, ds + g(x(T)) \right)$, where the infimum is over admissible controls on $[t, T]$ and $x$ is the trajectory with $x(t) = x$.
If the value function is continuously differentiable, it satisfies the equation.
Under standard hypotheses, such as $U$ compact and $f$, $L$, $g$ bounded and Lipschitz, it is the unique bounded uniformly continuous viscosity solution.
Example: for $x' = u$, $U = \R$, $L = u^2 / 2$ and $g(x) = x^2 / 2$, the minimum over $u$ is $-\frac{1}{2} (\partial_x V)^2$, and $V(t, x) = \dfrac{x^2}{2 (1 + T - t)}$ solves the equation.
Indeed $\partial_t V = \dfrac{x^2}{2 (1 + T - t)^2}$ and $\frac{1}{2} (\partial_x V)^2 = \dfrac{x^2}{2 (1 + T - t)^2}$, and $V(T, x) = x^2 / 2$.
Non-example: $W(t, x) = x^2 / 2$ satisfies the terminal condition but not the equation, since $\partial_t W - \frac{1}{2} (\partial_x W)^2 = -x^2 / 2 \ne 0$ for $x \ne 0$.
For infinite-horizon problems with discount rate $\rho > 0$ the equation becomes stationary: $\rho V(x) = \min_{u \in U} \{ \nabla V(x) \cdot f(x, u) + L(x, u) \}$.