least-squares problem with matrix

150 Views Asked by At

Consider the samples of vectors $(x_0,x_1,...,x_k)$ where $x\in \mathbb{R}^m$ and $(y_0,y_1,...,y_k)$ where $y\in \mathbb{R}^n$.
I need to find the matrix $A\in \mathbb{R}^{n\times m}$ of the following least-squares problem :

$\min_A S= \sum\limits_{i=0}^k (y_i-Ax_i)^T(y_i-Ax_i)$

To find the minimum, I differentiate $\frac{\partial S}{\partial A}=0=\sum\limits_{i=0}^k -2x_i^Ty_i +2x_i^TAx_i$

And finally : $\sum\limits_{i=0}^k x_i^TAx_i=\sum\limits_{i=0}^k x_i^Ty_i$

I am stuck here, how to find $A$ ?

1

There are 1 best solutions below

2
On BEST ANSWER

You can rewrite it as $$\sum\limits_{i=0}^k (y_i-Ax_i)^T(y_i-Ax_i)=\sum\limits_{i=0}^k ||y_i-Ax_i||_2^2=||Y-AX||_F^2$$ where X and Y are the matrices of the samples by column, and the $||\cdot||_F$ is the Frobenius norm. In this way you can transpose everything $$\dots=||Y^T-X^TA^T||_F^2$$ and you can treat it as a least squares minimization problem, that can be solved exploiting the pseudoinverse matrix. Remember that here $X^T,Y^T$ and $A^T$ play the roles respectively of the matrix of the linear system, constant term (one vector for each column) and the unknown (one vector for each column).