Optimal control with algeraic Riccati equation and infinite time

283 Views Asked by At

Motion of an object is described with \begin{cases} &\dot{y}_1 = ~~y_2, \\ &\dot{y}_2=-y_2+u, \end{cases} for $t\ge 0$, $\vec{y}(0)=\vec{y}_0$. The cost is $$ J(u)=\int\limits_0^\infty \left[ y_1(t)^2+y_2(t)^2+u(t)^2 \right]dt. $$

$u^0(t)=-K\vec{y}(t)$. We can form the algebraic Riccati equation: $$ A^*\Pi+\Pi A - \Pi B^* B \Pi + B^* B = O $$ (is this the correct associated Riccati Equation?). The problem is that I'm getting four sets of solutions for $\Pi$, which I think should not be the case.

Then, when $\Pi$ is obtained, we can find $K = -B^*\Pi$ and then solve for $\vec{y}$. Is this a correct approach? Another question is where do we use the functional in the whole process? Why is this functional given?

Also, how does one show that this system is open-loop controllable?

Would appreciate some help with this.

1

There are 1 best solutions below

0
On BEST ANSWER

This is also known as a Linear Quadratic Regulator. I will reformulate the problem to a more generic form,

$$ x = \begin{bmatrix} y_1 \\ y_2 \end{bmatrix}, \tag{1} $$

$$ \dot{x} = \underbrace{\begin{bmatrix} 0 & 1 \\ 0 & -1\end{bmatrix}}_A x + \underbrace{\begin{bmatrix} 0 \\ 1 \end{bmatrix}}_B u. \tag{2} $$

Now you want to find a $K$, such that,

$$ u(t) = -K\,x(t), \tag{3} $$

minimizes,

$$ J(u) = \int_0^\infty x^T \underbrace{\begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}}_Q x + u^T \underbrace{\left[1\right]}_R u \, dt. \tag{4} $$

If you allow me to make the assumption that we can also calculate this integral using quadratic terminal cost only depending of the terminal state, then this integral can also be written as,

$$ J(u) = \int_0^t x^T\!(\tau)\, Q\, x(\tau) + u^T\!(\tau)\, R\, u(\tau) \, d\tau + x^T\!(t)\, P\, x(t). \tag{5} $$

Matrix $P$ should remain the same, independent of the chosen $t$. If we now choose zero and an infinitesimal small time, $\delta$, for $t$, then both expressions for $J(u)$ should be the same. For this I will also use the substitution of equation $(3)$,

$$ x^T\!(0)\, P\, x(0) = \int_0^\delta x^T\!(\tau)\left(Q + K^T R\, K\right) x(\tau) \, d\tau + x^T\!(\delta)\, P\, x(\delta). \tag{6} $$

Since $\delta$ is so small we can approximate $x(t)$ for $0<t<\delta$ with a linear approximation. For this linear approximation I will again use equation $(3)$,

$$ x(t) \approx \left(I + t (A - B\,K)\right) x(0). \tag{7} $$

Using this approximation in equation $(6)$ and neglecting all quadratic or higher powers of $\delta$ yields,

$$ x^T\!(0)\, P\, x(0) = x^T\!(0)\left(Q + K^T R\, K\right) x(0) \, \delta + x^T\!(0)\left(P + \delta \left[(A - B\,K)^T P + P (A - B\,K)\right]\right) x(0). \tag{8} $$

Even though $\delta$ is very small, it is not zero. This equation should also hold for all possible $x(0)$. So this equation can only be true if,

$$ Q + K^T R\, K + (A - B\,K)^T P + P (A - B\,K) = 0. \tag{9} $$

If we are able to write this equation in the following form,

$$ (K - \hat{K})^T M (K - \hat{K}) = 0, \tag{10} $$

with $\hat{K}$ and $M$ for now unknown matrices, then the trivial solution would be $K=\hat{K}$. Expanding both equation $(9)$ and $(10)$ and equating them to each other yields,

$$ Q + A^T P + P\,A - K^T B^T P - P\,B\,K + K^T R\,K = \hat{K}^T M\,\hat{K} - K^T M\,\hat{K} - \hat{K}^T M\,K + K^T M\,K. \tag{11} $$

This can only be true in general if each similar term with $K$ is equal,

$$ \begin{align} Q + A^T P + P\,A = & \hat{K}^T M\,\hat{K}, \tag{12a} \\ B^T P = & M\,\hat{K}, \tag{12b} \\ P\,B = & \hat{K}^T M, \tag{12c} \\ R = & M. \tag{12d} \end{align} $$

The right hand sides of equations $(12b)$, $(12c)$ and $(12d)$ can be used to also express the right hand side of equation $(12a)$ as follows,

$$ \left(\hat{K}^T M\right) M^{-1} \left(M\,\hat{K}\right) = \hat{K}^T M\,\hat{K}. \tag{13} $$

Now using the left hand sides instead of the right hand sides of equations $(12)$ yields,

$$ P\,B\,R^{-1} B^T P = Q + A^T P + P\,A, \tag{14} $$

but this comes down to the exact same equation as the Continues Algebraic Riccati Equation.

I also have to note that the solution for $P$ also acts as a quadratic Lyapunov function of the form,

$$ V(x) = x^T P\,x. \tag{15} $$

This also brings me to the following point, namely that a Lyapunov function should always be positive for $x\neq 0$, which implies that $P$ should be positive definite. So all solutions you found, which are not positive definite can be discarded.

When you have found a positive definite $P$, then the corresponding "optimal" feedback control ($K$) can be found with rewriting equation $(12b)$ and using that $K=\hat{K}$,

$$ K = R^{-1} B^T P. \tag{16} $$