Optimal control involving derivative and square of bounded, decreasing control

52 Views Asked by At

This is a variant of Optimal control involving derivative of control.

We are given an ODE of the form $\dot y_t = (\dot \alpha_t - \alpha^2_t)y_t + \alpha^2_t$ with $y\in [0,\infty)$ and $t\in [0,T]$. The goal is to find a decreasing $\alpha^* : [0,T] \to [0,1]$, such that $-y_T^{\alpha^*}$ (the solution under this control) is maximized for any starting value $y_0$. Then, to get rid of the control derivative, it seems reasonable to set $Y = (y, \alpha)$, $\beta := \dot \alpha$ and instead look at the two-dimensional ODE $$\dot Y_t = \begin{pmatrix} (\beta - Y_{2,t}^2)Y_{1,t} + Y_{2,t}\\ \beta \end{pmatrix}.$$

Note, that $\beta \leq 0$ everywhere. The value function for this new problem is then $V(t,x_1) = \sup_{\beta\in \mathcal A(t)} -Y_{1,T}^{t,\beta}(x_1)$ (it does not depend on $x_2$).

This leads to the HJB equation

$$0 = - \partial_t V(t,x_1) - \sup_{b\in \mathbb R} \partial_{x_1} V(t,x_1)((b - x_2^2)x_1+x_2) = - \partial_t V(t,x_1) - \partial_{x_1} V(t,x_1) \inf_{b\in \mathbb R} ((b - x_2^2)x_1+x_2).$$

This seems non-sensical, though. It's clear that $\beta$ must be bounded if its assumed to be continuous, but even then, when we restrict to the infimum over $[\inf \beta, \sup \beta]$ instead of $\mathbb R$ this would suggest that $\dot \alpha^*$ is constant, which is much simpler then I would expect and completely ignores the $x_1,x_2$ terms.

Where am I going wrong with this?

PS: I have not adressed yet how to achieve the bounds $0,1$ on $\alpha$. My guess is that we can modify the value function by adding the term $-N \int_t^T \beta \,d\lambda + 1 - \alpha_T$ which forces $\alpha_0 = -\int_0^T\beta\,d\lambda + \alpha_T = 1$ as $N\to \infty$, and then $-\int_0^T \beta \,d\lambda \leq 1$ implying $0\leq 1 + \int_0^t \beta \,d\lambda = \alpha_t\leq 1$ for all $t\in [0,T]$.