SVD of a $3 \times 2$ matrix

2.4k Views Asked by At

This is not a homework question, but rather a practice one.

Find the singular value decomposition of $$A = \begin{bmatrix} 1 & 0 \\ 0 & -1 \\ 0 & 0\end{bmatrix}$$

I am attempting to find the matrices $U,V$ such as: $A = U\Sigma V^T$. I have found the following: the eigenvalues of $AA^T$ are $1,1,0$, and the eigenvectors $(x_1,x_2,x_3)$ that correspond to $1,1$ satisfy $(x_1,x_2,x_3) = (x_1,x_2,0)$ so one of them is $(1,0,0)$ and the other is $(0,1,0)$. The third eigenvector for the zero eigenvalue is $(0,0,1)$.

$U$ is a matrix whose columns must be the previous eigenvectors according to the definitions of SVD, so $$U = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{bmatrix}$$

The eigenvalues of $A^TA$ are $1,1$, whose corresponding eigenvectors are $(1,0),(0,1)$, and accroding to the definitions of SVD, $V$ is the matrix whose rows are those eigenvectors, so $$V = \begin{bmatrix} 1 & 0 \\ 0 & 1\end{bmatrix}$$

$\Sigma$ is the matrix with the square roots of the eigenvalues of the previous two matrices that are different than zero on the diagonal so $$\Sigma = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 0 & 0 \end{bmatrix}$$

I went according to the "algorithm" for finding the SVD, and I checked that the eigenvalues and eigenvectors I found are indeed correct (via wolfram). But when I ask wolfram for the SVD decomposition of the original matrix $A$, I get a different answer. When I also try to do the multiplication of the three matrices I have received, I do not receive the original matrix I started from. I have been trying to find the mistake in what I did but I have not noticed any.

Where was I wrong?

1

There are 1 best solutions below

0
On BEST ANSWER

From $\rm A^\top A = I_2$, we conclude that the singular values are $\sigma_1 = \sigma_2 = 1$ and that the right singular vectors are $\pm 1$ multiples of the vectors of the standard basis of $\mathbb R^2$. From $\rm A A^\top = {diag} (1,1,0)$, we (again) conclude that the singular values are $\sigma_1 = \sigma_2 = 1$ and that the left singular vectors are $\pm 1$ multiples of the vectors of the standard basis of $\mathbb R^3$.

One mistake you made was choosing the direction of the left and right singular vectors. Let us not choose the direction of the singular vectors. Hence,

$$\rm A = U \Sigma V^\top = \begin{bmatrix} s_1 & 0 & 0\\ 0 & s_2 & 0\\ 0 & 0 & s_3\end{bmatrix} \begin{bmatrix} 1 & 0\\ 0 & 1\\ 0 & 0\end{bmatrix} \begin{bmatrix} t_1 & 0\\ 0 & t_2\end{bmatrix}^\top = \begin{bmatrix} s_1 t_1 & 0\\ 0 & s_2 t_2\\ 0 & 0\end{bmatrix} = \begin{bmatrix} 1 & 0\\ 0 & -1\\ 0 & 0\end{bmatrix}$$

where $s_1, s_2, s_3 \in \{\pm 1\}$ and $t_1, t_2 \in \{\pm 1\}$. Thus, we have the constraints $\color{blue}{s_1 t_1 = 1}$ and $\color{blue}{s_2 t_2 = -1}$. Once we have chosen the values of parameters $s_1, s_2, s_3$, we use these two constraints to choose the values of parameters $t_1$ and $t_2$

$$t_1 = s_1 \qquad \qquad \qquad t_2 = - s_2$$

Thus, we have $\color{blue}{2^3 = 8}$ choices for the pair of matrices $\rm (U, V)$.