Optimal control and Value function

67 Views Asked by At

Let's consider this optimal control problem:

Minimize $-x(1)$,

subject to $dx(t)/dt=x(t)u(t)$ for almost every $t \in [0,1]$,

$x(0)=0$

among all the admissible controls $u:[0,1] \to [0,1]$ such that $u$ is Lebesgue measurable.

How can I compute the value function $V:[0,1] \times \mathcal{R} \to \mathcal{R}$?

1

There are 1 best solutions below

2
On

This can be solved using Dynamic Programming. Let $V(t,x)$ be the value function which must solve the Hamilton-Jacobi-Bellman equation

$$\min_{u\in[0,1]}\left\{\dfrac{\partial V(t,x)}{\partial t}+\dfrac{\partial V(t,x)}{\partial x}xu\right\}=0$$

together with the terminal condition $V(1,x)=-x$.

Let us consider now that $V(t,x)=p(t)x$, $p(1)=-1$. Then, we get that

$$\min_{u\in[0,1]}\left\{\dot{p}x+pxu\right\}=0.$$

So, if $px>0$, then $u=1$, and if $px\le 0$, then $u=0$.

Assume that $u=1$, then we have that $\dot p+p=0$ and we have that $p(t)=e^{-(t-1)}p(1)=-e^{1-t}$. This show that $V(t,x)=-xe^{1-t}$ when $p(t)x\le0$ or, equivalently, when $x\ge0$.

Assume that $u=0$, then we have that $\dot p=0$ and we have that $p(t)=c$ for some $c\in\mathbb{R}$. Since $p(1)=-1$, then $p(t)=-1$ and $V(t,x)=-x$ when $p(t)x>0$ or, equivalently, when $x<0$.

As a result, we have that

$$V(t,x)=\left\{\begin{array}{rl} -xe^{1-t},&\textrm{if } x\ge0,\\ -x,&\textrm{if } x<0\\ \end{array}\right.$$