I need to build the optimal controller, i.e. one that maximizes:
$J = \int_{0}^{t_f} f(u) \mathrm{d}t$
For the following time-dependent system:
$\dot{x} = g(x, u, t)$, $x(t) \geq l(t)\; \forall t \in [ 0, t_f]$
where $x(t) \in \mathbb{R}$ is the state, $u(t) \in \mathbb{R}$ the input, $t \in \mathbb{R}$ the time and $l(t) \in \mathbb{R}$ the time-varying lower bound for the state. Using the Pontryagin's maximum principle, I have defined the Hamiltonian as: $\mathcal{H} = \psi g(x, u, t) + f(u)$ and used the necessary condition $\dot{\psi} = - \frac{\partial \mathcal{H}}{\partial x}$. As you can see I have completely ignored the time-dependent inequality, so the solution I get is correct, but of course it doesn't enforce the lower bound $l$ on the state value.
How can I rewrite the hamiltonian so that also the inequality is considered?
I think we need to add $\lambda(t)(x(t)-l(t))$ to the Hamiltonian, where $\lambda(t)$ is a time-dependent Lagrange multiplier. Then we need to find a triplet of functions $x, u,\lambda$, such that the HJB-equations are satisfied and $\lambda(t) \geq 0$ and $\lambda(t)=0$ if $x(t)>l(t)$.