Applied Optimal Control - Minimize Integral Cost Function

Question

I am in the process of trying to solve an applied optimal control problem but I am not sure if my approach is correct. The problem is regarding a simple heating model between two rooms given by the system model $$\dot{\boldsymbol{x}} = \begin{bmatrix} -1.1 & 0.1 \\ 0.1 & -1.1 \end{bmatrix} \boldsymbol{x} + \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \boldsymbol{u}$$ and the room temperature starts at $\boldsymbol{x}(0)=[5,10]^T$ and the goal is to get the rooms close to the ambient temperature of $0$ by time $t_\text{f}=1$ using the least amount of energy. The cost function is given by

$$ J=\boldsymbol{x}^T(0)\boldsymbol{x}(0) + \int_0^1 \boldsymbol{u}^T\boldsymbol{u}~dt=x^2_1(0)+x^2_2(0) + \int_0^1 u_1^2(t)+u_2^2(t)~dt.$$ The control input $\boldsymbol{u}$ needs to be found that minimizes the cost function. Since the problem has a fixed time $t_\text{f}=1$ does that mean the problem is a "fixed time free state" optimization problem? Or is the state also fixed? Can this be solved with the Algebraic Ricatti Equation?

Answer 1

The given problem can be stated as

$$\text{min: }J = S(x(t_\text{f}),t_\text{f})+\int_{t_0}^{t_\text{f}}\mathcal{L}(t,x(t),u(t))~dt$$ $$\text{subject to: } \dot{x}=f(t,x(t),u(t)), x(t_0)=x_0, u\in \mathcal{U}.$$

For your problem, the final time is fixed and the final state is free. You solve this problem by the following method.

1. Step: Determine the Hamiltonian $$\mathcal{H}(t,x(t),u(t),\lambda(t))=\mathcal{L}(t,x(t),u(t))+\lambda^T(t)f(t,x(t),u(t))$$
2. Step: Use $\min_{u\in \mathcal{U}} \mathcal{H}$ (for constrained inputs) or $\dfrac{\partial H}{\partial u}=0$ (for unconstrained inputs) to obtain $u^*(t)=u^*(t,x^*(t),\lambda^*(t))$
3. Step: Plug $u^*(t)=u^*(t,x^*(t),\lambda^*(t))$ into the Hamiltonian to obtain $\mathcal{H}^*(t,x^*(t),\lambda^*(t))$
4. Step: Solve the coupled ODE system

$$\dot{x}^*(t) = \dfrac{\partial \mathcal{H^*}}{\partial \lambda^*}$$ $$\dot{\lambda}^*(t) = -\dfrac{\partial \mathcal{H^*}}{\partial x^*}$$

with the initial condition $x*(t=t_0)=x_0$ and the final conditions $$\left[\mathcal{H}^*+\dfrac{\partial S}{\partial t} \right]_{t_\text{f}}\delta t_\text{f}+\left[\dfrac{\partial S^*}{\partial x^*} -\lambda^*\right]^T_{t_\text{f}}\delta x_\text{f}=0.$$ If the final time / final state is fixed we have $\delta t_\text{f}=0$ / $\delta x_\text{f}=0$.

5. Step: Substitute $x^*(t)$ and $\lambda^*(t)$ from Step 2 to obtain the optimal control $u^*(t)$.

---

For the given problem we have.

1. $\mathcal{H}=u_1^2+u_2^2+\lambda_1u_1+\lambda_2u_2-1.1x_1\lambda_1+0.1x_2\lambda_1+0.1x_1\lambda_2-1.1x_2\lambda_2$

2. $$\dfrac{\partial \mathcal{H}}{\partial u}=\begin{bmatrix}2u_1+\lambda_1\\2u_2+\lambda_2 \end{bmatrix}=\begin{bmatrix}0\\0 \end{bmatrix}$$ $$\implies u^* = -[\lambda_1/2,\lambda_2/2]^T.$$

3. $\mathcal{H}^*=\lambda_1^2/4+\lambda_2^2/4-\lambda_1^2/2-\lambda_2^2/2-1.1x_1\lambda_1+0.1x_2\lambda_1+0.1x_1\lambda_2-1.1x_2\lambda_2$

4. $$\dfrac{d}{dt}\begin{bmatrix}x_1^*\\x_2^*\\\lambda_1^*\\\lambda_2^* \end{bmatrix} = \begin{bmatrix}-1.1 & 0.1 & -0.5 & 0.5\\0.1 & -1.1 & 0.0 & -0.5\\0.0 & 0.0 & -1.1 & 0.1\\ 0.0 & 0.0 & 0.1 & -1.1 \end{bmatrix} \begin{bmatrix}x_1^*\\x_2^*\\\lambda_1^*\\\lambda_2^* \end{bmatrix},$$ in which $\boldsymbol{x}^*(0)=[5,10]^T$ and $\boldsymbol{\lambda}^*(1)=2[x_1^*(1),x_2^*(1)]^T$.

5. Solve the previous system and use the states to replace $\boldsymbol{\lambda}^*(t)$ in $\boldsymbol{u}^*(t)=-0.5\boldsymbol{\lambda}^*(t)$