Using Least Squares to calculate a matrix in an equation.

Question

I have two sets of vectors $v_i$ and $w_i$, in some $d$ dimensional space. I know that $v_i \approx M \cdot w_i$ for all i. I.e., I know that the $v$ vectors are a linear transformation of the $w$ vectors.

I have more than $d$ vectors that match approximately. Is there a way to calculate the matrix $M$?

TLDR:

Given $X$ and $Y$, matrices with sizes $d \times n$ for $n \gg d$, can I calculate $M$ (a $d \times d$ matrix) such that $||Y - MX||$ is minimized?

Answer 1

If the matrix norm used in $||Y - M X||$ is the Hilbert-Schmidt norm, then, since $||Y - MX|| = ||X^T M^T - Y^T||$, the solution is $\underset{M}{\arg \min} ||X^T M^T - Y^T|| = ((X^T)^- Y^T)^T = Y X^{-}$, where $X^{-}$ is the pseudoinverse of $X$.

Answer 2

Assume that $\text{Rank}(X)=d$. Then differentiating $||MX-Y||_F^2$ w.r.t. $M$ yields

$$\frac{\partial}{\partial M}||MX-Y||_F^2=2X'(XM-Y)$$

The solution in the least-squares sense is obtained by setting the last eq. to $0$:

$$M_{LS}=(X'X)^{-1}X'Y=X^+Y$$