Problem: Given a discrete equation of state $$x^{k+1} = x^k -u^k,\ u^k \ge 0.$$ Our goal is to drive in $N$ steps the system to the origin, from $x^0$ and minimize the cost function $$J(x,u) := \sum_{k=0}^{N-1}u^k \ln{u^k}.$$
- Find the sufficient condition of the existence and the uniqueness of a solution.
- Find a close loop solution if possible.
My attempt:
- In my opinion, the condition should be $\nabla^2_u J(x,u) \succ0$ since $J(x,u)$ has a unique stationary point. But it seem obviously from the formula of the function $J$. So, I wonder that if I have any mistakes here.
- About the question 2, although I have read the theory about this topic a couple of times but I still have a lot of confusion. So, it will be great if you can help me in my first time studying this subject.
Convert your problem into a convex optimization problem. Let $u = [u_{0}, u_{1}, \dots, u_{N-1}] \in \mathbb{R}^{N}$. Then the problem can be written as,
$$ \begin{align} \min_{u} &&&\sum_{k=0}^{N-1} u_{k}\ln u_{k} \\ s.t. &&& u_{k} \geq 0 \quad \forall k\in [0,1,\dots, N-1] \\ &&& \mathbf{1}^{\mathsf{T}}u = x_{0} \end{align} $$
The last constraint is enforcing that the $x_N$ will be at the origin. Now you can use all the tools from optimization to answer if this problem has a solution, i.e., does the sum of strictly convex functions with affine inequality and equality constraints have a solution? For uniqueness, consider what would happen if you had an optimal solution $u^{*}$ and you "swap" the values of $u^{*}_{1}$ and $u^{*}_{2}$. Would this solution also be optimal? Lastly, you can use methods in optimization to find the optimal control input.