Let's consider the following deterministic constrained optimisation problem, where $c(t)$ is the control, and $x(t)$ and $y(t)$ are the state variables: \begin{align} J(t) = \inf_{c(t)} \ &\int_0^\infty e^{-\delta u} \left( x(u)^2 + \lambda y(u)^2 \right) du \\ s.t. \ &c(t) \geq - \xi \end{align} where the dynamics for the states are: \begin{align} x'(t) &= c(t) - y(t) - a \\ y'(t) &= \delta y(t) - b x(t) \end{align} where $x_0$ is known and the following transversality condition holds: $\lim_{T \to \infty} E_t [e^{-\delta T} y(T)]=0$
I want to solve this optimisation problem. First, I write the extended Hamiltonian: \begin{align} \mathcal{H} = x(t)^2 + \lambda y(t)^2 + J_{x}(t) ( c(t) - y(t) - a ) + J_{y}(t) \left( \delta y(t) - b x(t) \right) + \mu_{c}(t) (c(t) + \xi) \end{align} where $J_{x}(t)$ and $J_{y}(t)$ denote the derivatives of the value function with respect to the states (i.e. the co-state variables), and $\mu_{c}(t)$ is the Lagrange multiplier.
Next, I write the three optimality conditions: \begin{align} 0 &= \mathcal{H}_{c}(t) \\ J_{x}(t) &= - \mu_{c}(t) \nonumber \end{align} \begin{align} 0 &= \mathcal{H}_{x}(t) + \frac{dJ_{x}(t)}{dt} - \delta J_{x}(t) \\ 0 &= 2 x(t) - J_{y}(t) b + \frac{dJ_{x}(t)}{dt} - \delta J_{x}(t) \nonumber \end{align} \begin{align} 0 &= \mathcal{H}_{y}(t) + \frac{dJ_{y}(t)}{dt} - \delta J_{y}(t) \\ 0 &= 2 \lambda y(t) - J_{x}(t) + \delta J_{y}(t) + \frac{dJ_{y}(t)}{dt} - \delta J_{y}(t) \nonumber \\ 0 &= 2 \lambda y(t) - J_{x}(t) + \frac{dJ_{y}(t)}{dt} \nonumber \end{align} with the complementary slackness conditions: \begin{align} 0 &= \mu_{c}(t) (c(t) + \xi) \\ 0 &= J_{x}(t) (c(t) + \xi) \nonumber \end{align} \begin{align} 0 \leq \mu_{c}(t) \\ 0 \geq J_{x}(t) \nonumber \end{align} I have three questions, and I would appreciate even a partial answer if you are unable to answer all three of them:
- Is my slackness condition that says that the co-state of x must be negative ($0 \geq J_{x}(t)$) correct?
- I proceed to solve the problem as follows: first I solve it in in the region where $J_{x}(t) = 0$, then I solve it in the region where $c(t) = - \xi$. This means that the solution for the region where the constraint is not binding is identical to the one of a problem where there is no constrain on the control at all. But intuitively it does not make sense that the constraint has no effect on the policy in the unconstrained region. What is your explanation? Do you have some references that explain this?
- If I wanted to solve this with the HJB equation, would this formulation for the HJB be correct? $$HJB: x(t)^2 + \lambda y(t)^2 + J_{x}(t) ( c(t) - y(t) - a ) + J_{y}(t) \left( \delta y(t) - b x(t) \right) - \delta J(t) + \mu_{c}(t) (c(t) + \xi) $$