I have a question regarding least square estimation and state trajectory.
I have the problem:
but I don't understand how I can rewrite the subject to? I believe that you can set the minimization to the form: $$||y - Cx||_{2}^{2}$$
and then just expand that to:
$$y^{T}y - 2y^{T}Cx + x^{T}C^{T}Cx$$
is that correct?
But I don't know what to do with the subject to, to finally reach the x(0) expression.
Grateful for all the help I can get.
The key is that we want to reframe this problem as an unconstrained least squares optimization. Currently, the problem is framed as a search for $m$ vectors $x_1,\dots,x_m$ satisfying the constraint that $x_{i+1} = Ax_i$ for $i = 1,\dots,m$. To that end, we can note that as soon as we decide on an initial state $x_1$, the remainder of the $x_i$ are completely determined, and there are no limitations on our choice of $x_1$. In particular, for any $x_1$, we have $$ x_2 = Ax_1, \quad x_3 = Ax_2 = A^2x_1, \quad x_4 = Ax_3 = A^3 x_1, \dots, $$ and so forth, which is to say that $x_{i} = A^{i-1}x_1$. Thus, we can rewrite the problem as the following unconstrained minimization: $$ \min_{x_1} \frac 12 \sum_{i=1}^m \|Cx_i - y_i\|^2 = \min_{x_1} \frac 12 \sum_{i=1}^m \|CA^{i-1}x_1 - y_i\|^2 = \frac 12 \min_{x_1} \sum_{i=1}^m \|CA^{i-1}x_1 - y_i\|^2. $$ In order to frame this in terms of traditional least squares, it is helpful to represent this minimization in the form $\min_{x_1} \sum_i \|CA^{i-1}x_1 - y_i\|^2$ in the form $\min_{x_1} \|Mx - b\|^2$ for some matrix $M$ and vector $b$. We can do this nicely with the help of block-matrix operations. We have \begin{align} \sum_{i=1}^m \|CA^{i-1}x_1 - y_i\|^2 &= \left\|\pmatrix{Cx_1\\ CAx_1 \\ \vdots \\ CA^{m-1}x_1} - \pmatrix{y_1\\ y_2\\ \vdots \\ y_m}\right\| = \left\| \pmatrix{C\\CA\\ \vdots \\ CA^{m-1}} x_1 - \pmatrix{y_1\\ y_2\\ \vdots \\ y_m} \right\|, \end{align} so that our optimization is in the form $\min_{x_1} \|Mx_1 - b\|^2$ with $$ M = \pmatrix{C\\CA\\ \vdots \\ CA^{m-1}}, \quad b = \pmatrix{y_1\\ y_2\\ \vdots \\ y_m}. $$ Perhaps you can take it from there.