Physical Meaning Behind Matrix Factorization

Question

As we all know, Matrix Factorization is an effective method to do rating prediction jobs in recommender systems. Thanks to the work of Yahuda Koren. My question is why MF can do this job ? What's the physical meaning behind it? Waiting for your kindly reply....

Answer 1

The Singular Value Decomposition is one of the most useful matrix factorizations, and is used often in the application you mentioned.

A great way to understand the 'meaning' of the SVD is to think of each of the three factors $U,\Sigma, $ and $V^*$ as a step in a process which takes $x$ to $Mx$ (where $M = U\Sigma V^*$).

First, recall that $V$ and $U$ are both unitary. This means that magnitudes are preserved under the transformation (i.e. $\langle Ux,Uy \rangle = \langle x,y \rangle$ for all vectors $x$ and $y$). Also note that $\Sigma$ is a diagonal matrix of scalars, and so only affects magnitude and does not rotate vectors it operates on.

Now think about what happens to a vector $x \in \mathbb{R}^n$.

First, $V^* \in \mathbb{R}^{n\times n}$ rotates the vector (rotations are the types of linear transformation which preserve magnitude) in $n$-dimensional space.

Then $\Sigma$ scales the vector $V^*x$.

Finally, $U\in \mathbb{R}^{m\times m}$ rotates the vector in $m$-dimensional space.

Wikipedia has a great animation which reinforces this process.