When is the Hamiltonian constant?

Question

In the context of optimal control, when is the Hamiltonian constant?

I know that, generally, when $H$ is not explicitly a function of time, it is going to be a constant, but I just did a problem where the Hamiltonian switched from a constant value to another constant value at some later time, like a piecewise function. So, can anyone clarify when the Hamiltonian is truly a constant, and when it is a piecewise function?

Answer 1

We have to be more precise when talking about something being constant. A constant is a number or vector.

In optimal control theory, the Hamiltonian $\mathcal{H}$ can additionally be a function of $x(t)$, $u(t)$ and $\lambda(t)$. Hence, it is not constant. If you are only considering invariance with time then

$$\dfrac{d\mathcal{H}}{dt}=\dfrac{\partial \mathcal{H}}{\partial x}\dfrac{\partial x}{\partial t}+\dfrac{\partial \mathcal{H}}{\partial u}\dfrac{\partial u}{\partial t}+\dfrac{\partial \mathcal{H}}{\partial \lambda}\dfrac{\partial \lambda}{\partial t}$$.

Answer 2

Constant Hamiltonian in Optimal Control Theory are related to the Beltrami Identity appearing in Calculus of Variations.

In Calculus of Variations, if the Lagrangian $ \mathcal{L} $ don't explicetly depend on time such that $ J = \int \mathcal{L}(x,\dot{x}) dt $ then the Beltrami Identity $ \mathcal{L} - \frac{\partial \mathcal{L}}{\partial \dot{x}_{\alpha}} \dot{x}_{\alpha} = C $ holds where $ C $ is a constant.

In Optimal Control Theory the Lagrangian are given by $ \mathcal{L} = L(t, \mathbf{x},\mathbf{u}) + \dot{\phi}(t,\mathbf{x} ) + \boldsymbol{\lambda} \left( \mathbf{f}(t,\mathbf{x},\mathbf{u}) - \mathbf{\dot{x}} \right) $ where $ L $ are Running Cost, $ \phi $ are terminal cost an $ f $ are system dynamics. If all those three quantities don't explicetly depend on time then neither the Lagrangian $ \mathcal{L} $ does and the Costate Equation can be re-written using the Beltrami Identity, and it will collapses into $ H=C $

$ \mathcal{L} - \frac{\partial \mathcal{L}}{\partial \dot{x}_{\alpha}} \dot{x}_{\alpha} = C $

$ \left\{ H + \frac{d \phi}{dt} - \lambda_{\theta}\dot{x}_{\theta} \right\} - \frac{\partial }{\partial \dot{x}_{\alpha}} \left\{ H + \frac{d \phi}{dt} - \lambda_{\theta}\dot{x}_{\theta} \right\} \dot{x}_{\alpha} = C $

$ H + \frac{\partial \phi}{d x_{\beta}} \dot{x}_{\beta} - \lambda_{\theta}\dot{x}_{\theta} - \frac{\partial }{\partial \dot{x}_{\alpha}} \left\{ \frac{\partial \phi}{d x_{\beta}} \dot{x}_{\beta} - \lambda_{\theta}\dot{x}_{\theta} \right\} \dot{x}_{\alpha} = C $

$ H + \frac{\partial \phi}{d x_{\beta}} \dot{x}_{\beta} - \lambda_{\theta}\dot{x}_{\theta} - \frac{\partial \phi}{d x_{\alpha}} \dot{x}_{\alpha} + \lambda_{\alpha} \dot{x}_{\alpha} = C $

$ H = C $

$ $