How to solve $\frac{\partial V}{\partial t} + x + \frac{\partial V}{\partial x}- \frac{1}{2} \frac{1}{\left(\frac{\partial V}{\partial x}\right)} = 0$

94 Views Asked by At

I'm solving this problem of optimal control using the dynamic programming: $$ \begin{cases} \min \displaystyle \int_0^2(x-u)dt + x(2) \\ \dot x = 1+u^2 \\ x(0) = 1 \end{cases} $$

Then solving the Bellman-Hamilton-Jacobi equation I found the following PDE: $$\frac{\partial V}{\partial t} + x + \frac{\partial V}{\partial x}- \frac{1}{4} \frac{1}{\left(\frac{\partial V}{\partial x}\right)} = 0$$

The problem gives an hint: In order to solve BHJ equation, we suggest to find the solution in the family of functions $\mathcal{F} = \{V(t,x) = A +Bt + Ct^2 + D\log(3-t) + E(3-t)x\}$ where $A,B,C,D,E$ are all real costants.

My question is:

How can someone derives that all the solution of the BHJ equation are in that family of functions? In other words, how could I manage to solve the BHJ without any hint?

1

There are 1 best solutions below

1
On BEST ANSWER

I find that this problem can be more easily solved using Pontryagin's maximum principle, which gives the following Hamiltonian

$$ H(t,x,\lambda) = x - u + \lambda\,(1 + u^2), $$

such that

$$ \dot{\lambda} = -1, \quad u = \frac{1}{2\,\lambda} $$

and from the terminal cost is follows that $\lambda(2) = 1$. In this case solving for the co-state as a function of time is easy, namely $\lambda(t) = 3 - t$ and thus

$$ u(t) = \frac{1}{2\,(3 - t)}. \tag{1} $$

Note that this solution is still independent of $x(0)$.


When formulating the PDE one also obtains that

$$ u = \frac{1}{2\,V_x}, \tag{2} $$

with $V_x$ shorthand notation for the partial derivative of $V(t,x)$ with respect to $x$. Equating $(2)$ to $(1)$ yields

$$ V_x = 3 - t. \tag{3} $$

Therefore, the final expression for $V(t,x)$ should be of the form

$$ V(t,x) = x\,(3 - t) + U(t), \tag{4} $$

with $U(t)$ a yet unknown function of only $t$ and no $x$. Substituting $(4)$ together with $(3)$ in the PDE yields

\begin{align} 0 &= -x + \dot{U} + x + 3 - t - \frac{1}{4} \frac{1}{3 - t}, \\ &= \dot{U} + 3 - t - \frac{1}{4} \frac{1}{3 - t}. \end{align}

From this it becomes hopefully clear where the function family comes from. Namely, the last term is derived from $(3)$, the second order polynomial comes from integrating $t - 3$ and the logarithm from integrating $(3 - t)^{-1}$.

I am not sure how one could spot this family of functions without Pontryagin's maximum principle. Though, I suspect this is also why the exercise gave the function family, because solving nonlinear PDE's is hard.