Explain the geometrical meaning of Singular Value Decomposition (SVD)

1.8k Views Asked by At

Suppose you have a 2x2 real-valued matrix, $\mathbf{M}$. If you perform a singular value decomposition (SVD), then Wikipedia and the internet tell me that this can be understood geometrically as a decomposition of $\mathbf{M}$ into a rotation, scaling and second rotation of the form:

$$\mathbf{M} = \mathbf{U S V}^\mathrm{T}$$

where the $\mathrm{T}$ denotes transpose. Here, $\mathbf{V}^\mathrm{T}$ is the first rotation, $\mathbf{S}$ is the scaling matrix and $\mathbf{U}$ is the last rotation.

Wikipedia has this nice little graphic:

enter image description here

So, what I am trying to do is essentially reproduce this graphic in MATLAB. I have a unit circle represented as:

$$\mathbf{C} = \begin{bmatrix} 1&0 \\ 0 & 1 \end{bmatrix}$$

This can be plotted in MATLAB like this:

enter image description here

I then have some arbitrary shearing matrix which is given by:

$$\mathbf{M} = \begin{bmatrix} 0.5&4 \\ 1 & 1.5 \end{bmatrix}$$

which represents an ellipse which can also be plotted in MATLAB:

enter image description here

So far so good because I can reproduce the top two images in the Wikipedia article.

If I do the SVD for $\mathbf{M}$ I can get:

$$\mathbf{U} = \begin{bmatrix} -0.9239&-0.3827 \\ -0.3827 & 0.9239 \end{bmatrix}$$

$$\mathbf{S} = \begin{bmatrix} 4.3523&0 \\ 0 & 0.7467 \end{bmatrix}$$

$$\mathbf{V} = \begin{bmatrix} -0.1941&-0.9810 \\ -0.981 & -0.1941 \end{bmatrix}$$

Now, can someone explain what I need to do to produce the bottom two images from the Wikipedia article?

I thought that I would just be able to plot the columns of $\mathbf{V}^\mathrm{T}\mathbf{C}$ to get the rotated unit vectors for the bottom left image like so:

enter image description here

Next, I would plot the ellipse of $\mathbf{SC}$ to get the bottom right stretched image and I can apply $\mathbf{SV}^\mathrm{T}$ to the unit vectors to get:

enter image description here

This is clearly wrong though because the major axis of the dashed ellipse is much longer than the major axis of the solid ellipse representing $\mathbf{M}$. If I rotate the dashed ellipse, it will not match the solid ellipse.

What am I doing wrong?