I am trying to minimize an time-integral of a linear function with respect to differential equations. The problem is formally defined as follows:
Given $\lambda< \mu_1, \mu_2$ fixed parameters(thus $x(s)$ will hit 0 at some point in time and stay there) and $c_1, c_2$ time independent, fixed cost coefficients :
$v(x_1, x_2) = \min_{u,p} \int\limits_T \ [c_1 X_1(s) + c_2 X_2(s)]ds$ \
subject to
$\dot{X_1}(s) = p(s)\lambda - u(s)\mu_1$;
$\dot{X_2}(s) = (1- p(s))\lambda - (1-u(s))\mu_2$;
$ X_i(s) = x_i$
$0 \leq u(s) \leq 1$
$0 \leq p(s) \leq 1$
Thus $u(s), p(s)$ are the controls or decision variables. What would be the methodology to follow in this case?
If I understood properly, for this system to reach optimality HJB equation must be satisfied :
$\min_{u,p} (p(s)\lambda - u(s)\mu_1)\frac{\partial v}{\partial x1} + (1- p(s))\lambda - (1-u(s))\frac{\partial v}{\partial x2} + c_1 x_1 + c_2 x_2 = 0 $
But without knowing the exact form of $v$ I can not check whether the equation above holds or not. I guess one way to progress is to guess optimal $v$, but then again how such an argument would work without being circular.
Any help, references, comments appreciated... Thanks in advance..
Use PMP (Pontryagen maximum principle). Pontryagen function (hamiltonian) in your problem is
$$ H = (c_1x_1+c_2x_2) + \phi_1(\lambda p + \mu_1 u) + p_2(\lambda (1-p) + \mu_2 (1-q)) $$
Here $\phi_1$ and $\phi_2$ are the adjoint variables to $x_1$ and $x_2$. So
$$ \dot\phi_i = -\frac{\partial}{\partial x_i}H $$
$$ \dot\phi_1 = c_1\ \ \mbox{ and }\ \ \dot \phi_2=c_2 $$
and control variables $u$ and $p$ can be found from maximum principle:
$$ H\to\max_{u,p} $$
i.e. $p=sign(\phi_1-\phi_2)$ and $u=sing(\mu_1\phi_1 - \mu_2\phi_2)$.As a result you will have simple ODE system with 4 variables $x_1$, $x_2$, $\phi_1$ and $\phi_2$. It can be solved easely by standard methods