Find optimal solution for OCP

Question

There is OCP:

Maximize $$\int_0^T (x^2-4x) dt$$ Subject to $$\frac{dx}{dt} = u$$ $$x(0) =0$$ $$x(T) free$$ $$u\in[0,1]$$ The answer from book is $$for\quad T \leq 6\quad u*=0$$ $$for\quad 6 \leq T\quad u*=1$$

How to solve it?

Hamiltonian is $$H=x^2-4x+\psi u$$ $$u* = argmax(H)$$ $$u*=0 \quad \psi<0 $$ $$u*=1 \quad \psi>0 $$ $$u* \in [0, 1] \quad \psi=0$$

Adjoint system is $$d\psi/dt = -2x+4$$ $$\psi(T)=0$$

There are 3 cases:

1. $u*=1, \psi>0$

then $$x=t, \psi'=-2t+4,$$$$\psi =-t^2+4t+T^2-4T>0$$ $$(T - t) (t + T - 4)>0$$ 2. $u*=0, \psi<0$

then $$x=0, \psi'=4,$$$$\psi =4t-4T<0$$ $$t<T$$ 3. $$\psi=0$$ ?

is it correct to use PMP?

are there any mistakes in what was written? how to finish it?

Answer 1

• "Is it correct to use PMP?" Sure, this is indeed typically the type of problems for which you want to apply Pontryagin's maximum principle. It is made exactly for that.
• "Are there any mistakes in what was written?" Your setup, your Hamiltonian and your adjoint system all look OK. However, there is a problem starting with your sentence "There are 3 cases". While it is true that there are 3 dynamics (depending on the sign of $\psi$), you have no guarantee that you will stay within the same regime for all $t \in [0,1]$. In fact, the alternative depends on the sign of $\psi(t)$ since $\psi$ depends on $t$ through $\dot{\psi} = - 2 x + 4$. So an "optimal" solution could switch between a time interval where $\psi(t) < 0$, then another one where $\psi(t) > 0$, then another ... and so on. Determining the "geometry" of the optimal solution is the most difficult part in applying the PMP.
• "What about $\psi(t) = 0$?" Imagine that $\psi(t) = 0$ on some small interval $[t_0,t_1]$. Then $\dot\psi(t) = 0$ on this interval, so $x(t) = 2$ on this interval, so $u(t) = 0$. So you know everything about it. Moreover, during this interval, the integral $\int_{t_1}^{t_2} x^2 - 4 x = - 4 (t_2-t_1) < 0$. Hence such a region makes the objective diminish. So you never have such an "arc", as you could remove it and obtain a better solution.
• "How to finish it?" Well, you need to tackle the difficulty mentioned above. Let me give you a few pointers. The most important unknown value is $\psi(0)$. For any fixed $\psi_0 \in \mathbb{R}$, we can look at the solution of $\psi(0)=\psi_0$, $x(0) = 0$, $\dot\psi = -2x+4$, $\dot x = u$, $u = \operatorname{argmax} H$. This is a forward Cauchy problem so you will be able to compute its solution. You will have to be very careful along the way to determine the type of "arcs" you are encountering. Then, you will need to adjust $\psi_0$ such that $\psi(T) = 0$.
• I would advise that you plot a few graphs. For example, you know that $\ddot \psi = - 2 u \leq 0$ on $[0,T]$ and that $\dot{\psi}(0)= -2x(0)+4 = 4 > 0$. This should help you visualize what kind of succession of "regions" or "arcs" you can have.