Guess and verify: verification theorem for Hamilton-Jacobi-Bellman equation

351 Views Asked by At

Let $t \in [0,\infty)$ denote time, $x(t) \in X \subset \mathbb R_+$ the state and $u(t) \in \mathbb R_+$ the control. Consider the following optimal control Problem \begin{align} &V(x_0) = \max_{u(t)}\int_0^\infty{e^{-rt}\ln(u(t))dt}\\[2mm] \text{s.t.} \quad &\dot x(t) = -u(t) \tag{O}\\ &x_0 = x(0) > 0. \end{align} where $r > 0$. I claim that the feedback law $\phi(x(t)) = rx(t)$ for all $t \in [0,\infty)$ is the optimal solution.

Proposition There exists an optimal solution for (O) characterized by \begin{align} &V(x) = \frac{1}{r}(\ln(rx) - 1),\\ &\phi(x) = rx. \end{align}

  • How can I prove that?

I know that $\phi(x)$ must satisfy the Hamilton-Jacobi-Bellman (HJB) equation for all $x \in X$ \begin{align} rV(x) =& \max_{u}\{\ln(u) - V'(x)u\}\\ =& \ln\left(\frac{1}{V'(x)}\right) - 1\\ =& \ln(rx) - 1. \end{align} Check. I also know that the transversality condition \begin{align} \lim_{t \to \infty} e^{-rt}V(x(t)) = 0 \end{align} must be satisfied. The solution of $\dot x(t) = -rx(t)$ is given by \begin{align} x(t) = x_0e^{-rt}. \end{align} It follows \begin{align} &\lim_{t \to \infty} e^{-rt}V(x(t))\\ =&\lim_{t \to \infty} e^{-rt}\frac{1}{r}(\ln(r x_0e^{-rt})-1)\\ =&\lim_{t \to \infty} \frac{\ln(r x_0) - rt - 1}{e^{rt}r}\\ =&\lim_{t \to \infty} \frac{r}{e^{rt}r^2}\\ =&0. \end{align} Check. Are we done? Is this procedure sufficient to show that $\phi(x)$ is indeed optimal?