Convert least squares problem $\| ZXd - y\|^2$ into form $\|A\text{vec}(X) - b \|^2$

398 Views Asked by At

I have a least squares problem in this form $$ \| ZXd - y \|^2 $$ where $X$ is rectangular matrix of my variables, $Z$ - rectangular matrix of constants, $d$ and $y$ just vectors.

And i want to convert this problem into more canonical form $$\|A\text{vec}(X) - b \|^2$$

where $\text{vec}(X)$ stacks columns of matrix $X$ into one big vector

So my question is - how i can obtain new matrix of coefficients $A$ and vector $b$?

This wiki page provides some ideas of obtaining $A$ matrix but i still have not idea how to obtain vector $b$

PS Sorry for my English.

1

There are 1 best solutions below

0
On BEST ANSWER

Yes, you can rewrite it with vectorization and Kronecker products:

$$\|{\bf ZXd - y}\|^2 \to \|{\bf M_1M_2}\text{vec}({\bf X}) - \text{vec}({\bf y})\|^2$$

where $$\cases{{\bf M_1} = \text{Multiplication from left by }{\bf Z}\\{\bf M_2} = \text{Multiplication from right by }{\bf d}}$$

Now let us construct these matrices

$$ {\bf AXB} \to ({\bf B}^T\otimes {\bf A})\text{vec}({\bf X})$$

Now for $\bf M_1$: $\cases{\bf A=Z\\\bf B=I} \to {\bf M_1} = ({\bf Z}^T\otimes {\bf I})$.

And for $\bf M_2$: $\cases{\bf A=I\\\bf B=d} \to {\bf M_2} = ({\bf I}^T\otimes {\bf d})$.

So our vectorized system becomes: $$\begin{align*}\min_{\text{vec}({\bf X})}&\|{\bf M_1 M_2 } \text{vec}({\bf X})-\text{vec}({\bf y})\|^2= \\\min_{\text{vec}({\bf X})}&\|\underset{\text{The }{\bf A}\text{ you are looking for}}{\underbrace{({\bf Z}^T\otimes {\bf I})({\bf I}^T\otimes {\bf d})}{ \text{vec}({\bf X})}}-\text{vec}({\bf y})\|^2\end{align*}$$