Changing sum of vector products to a matrix

Question

I always struggle when trying to convert vectors to matrices. Example Analytic solution for matrix factorization using alternating least squares

$$\begin{align}\frac{\partial C}{\partial x_u} &= -2\sum_ic_{ui}(p_{ui}-\color{red}{x_u^Ty_i})y_i+2\lambda x_u\\&=-2\sum_ic_{ui}(p_{ui}-\color{red}{y_i^Tx_u})y_i+2\lambda x_u\\&=-2Y^TC^up(u)+2Y^TC^uYx_u+2\lambda x_u \text{ Eq.(1)}\end{align}$$ where each row of matrix $Y\in \mathbb{R}^{n\times f}$ is $y_i^T$, the diagonal matrix $C^u\in\mathbb{R^{n\times n}}$ has coefficient $c_{ui}$ in row/column $i$, and vector $p(u)\in\mathbb{R^n}$ contains element $p_{ui}$ in row $i$.

How do I get the 3rd line from the 2nd line ? Is there a cheat sheet I can refer to or a video or a book or paper that I can read that will tell me the tips and tricks of upgrading a vector to a matrix ??????????? I am stupid and this is extremely important SO PLEASE HELP ME.

Answer 1

You can apply the following three facts:

1. For vectors $y_i$ and scalars $a_i$ it holds $$ \begin{matrix}|\\y_1\\|\end{matrix}\cdot a_1+\begin{matrix}|\\y_2\\|\end{matrix}\cdot a_2+\ldots+\begin{matrix}|\\y_n\\|\end{matrix}\cdot a_n= \begin{bmatrix} \begin{matrix}|\\y_1\\|\end{matrix} &\begin{matrix}|\\y_2\\|\end{matrix} & \ldots & \begin{matrix}|\\y_n\\|\end{matrix} \end{bmatrix}\cdot \begin{bmatrix} a_1\\a_1\\\vdots\\a_n \end{bmatrix}. $$
2. For scalars $c_i$ and $q_i$ it holds $$ \begin{bmatrix} c_1q_1\\c_2q_2\\\vdots\\c_nq_n \end{bmatrix}=\begin{bmatrix}c_1 &&&\\&c_2&&\\&&\ddots&\\&&&c_n\end{bmatrix}\begin{bmatrix}q_1\\q_2\\\vdots\\q_n\end{bmatrix}. $$
3. For vectors $y_i$ and a vector $x$ it holds $$ \begin{bmatrix}y_1^Tx\\y_2^Tx\\\vdots\\y_n^Tx\end{bmatrix}= \begin{bmatrix}\begin{matrix}-&y_1^T &-\end{matrix}\\ \begin{matrix}-&y_2^T &-\end{matrix}\\ \vdots\\ \begin{matrix}-&y_n^T &-\end{matrix} \end{bmatrix}\begin{bmatrix}\begin{matrix}|\\x\\|\end{matrix}\end{bmatrix} $$ Proof: trivial by definition of matrix multiplication.

Apply the first fact to $a_i=c_{ui}(p_{ui}-\color{red}{x_u^Ty_i})=c_{ui}\cdot (p_{ui}-y_i^Tx_u)$, the second one to $q_i=p_{ui}-y_i^Tx_u$ etc