Singular Value Decomposition gives a row permutated matrix

70 Views Asked by At

I want to find the SVD of the following matrix. $$A = \begin{bmatrix}1&2\\2&2\\2&1\end{bmatrix}$$

The singular values of the above matrix are $\sqrt{17}$,$1$. The following results are posted here in order to make this question shorter.

$$AA^T=\begin{bmatrix}5&6&4\\6&8&6\\4&6&5\end{bmatrix}$$ Eigen values of $AA^T$=$\{0,1,17\}$

Eigen vectors of $AA^T$=$\{\begin{bmatrix}-2\\3\\-2\end{bmatrix},\begin{bmatrix}1\\0\\-1\end{bmatrix}, \begin{bmatrix}3\\4\\3\end{bmatrix}\}$

$$A^TA=\begin{bmatrix}9&8\\8&9\end{bmatrix}$$ Eigen values of $A^TA$=$\{1,17\}$

Eigen vectors of $A^TA$=$\{\begin{bmatrix}1\\-1\end{bmatrix},\begin{bmatrix}1\\1\end{bmatrix}\}$

Therefore, we can write the U,V,W matrices as follows.

$$U=\begin{bmatrix}\frac{3}{\sqrt{34}}&\frac{1}{\sqrt{2}}&\frac{-2}{\sqrt{17}}\\ \frac{4}{\sqrt{34}}&0&\frac{3}{\sqrt{17}}\\ \frac{3}{\sqrt{34}}&\frac{-1}{\sqrt{2}}&\frac{-2}{\sqrt{17}}\end{bmatrix}$$

$$W=\begin{bmatrix}\sqrt{17}&0\\0&1\\0&0\end{bmatrix}$$ $$V=V^T=\begin{bmatrix}\frac{1}{\sqrt{2}}&\frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}&\frac{-1}{\sqrt{2}}\end{bmatrix}$$

My problem is that why in here, $$A\neq UWV^T$$ $$UWV^T=\begin{bmatrix}2&1\\2&2\\1&2\end{bmatrix}$$ which is a row permuted version of A.

This problem won't occur if I multiplied the eigenvector of $AA^T$ corresponding to the eigenvalue of $1$ by $(-1)$.(i.e; by using $\begin{bmatrix}-1\\0\\1\end{bmatrix}$ instead of $\begin{bmatrix}1\\0\\-1\end{bmatrix}$). Both of those eigenvectors are correct for the eigenvalue $1$ of that matrix. So, my problem is how do we identify such occasions when obtaining the SVD?

1

There are 1 best solutions below

0
On BEST ANSWER

With the help of Lutz Lehmann's comment above I figured out the way to obtain the SVD without worrying about the problem I previously faced. I am posting my answer here so that it would be a help to anyone who has the same problem.

In the SVD, we express the matrix A as a product of 3 matrices which I am denoting as U, W, $V^T$. In this less problematic method, what we have to first do is obtaining the $A^TA$.

$$A = \begin{bmatrix}1&2\\2&2\\2&1\end{bmatrix}$$ $$\implies A^TA=\begin{bmatrix}9&8\\8&9\end{bmatrix}$$


Then we obtain the eigenvalues and corresponding eigenvectors of $A^TA$ as follows.

$det(A^TA-\lambda I)=det(\begin{bmatrix}9-\lambda&8\\8&9-\lambda\end{bmatrix})=0$

$\implies (1-\lambda)(17-\lambda)=0$

$\therefore \lambda=\{17,1\}$

When $\lambda = 17,$

eigenvector = $\begin{bmatrix}1\\1\end{bmatrix}$

$\therefore$ normalized eigenvector when ($\lambda = 17$) = $\begin{bmatrix}\frac{1}{\sqrt{2}}\\\frac{1}{\sqrt{2}}\end{bmatrix}=v_1$

When $\lambda = 1,$

eigenvector = $\begin{bmatrix}1\\-1\end{bmatrix}$

$\therefore$ normalized eigenvector when ($\lambda = 1$) = $\begin{bmatrix}\frac{1}{\sqrt{2}}\\\frac{-1}{\sqrt{2}}\end{bmatrix}=v_2$


The corresponding singular value would be $\{\sigma_1=\sqrt{17},\sigma_2=1\}$.

Using the above results, we can write W and V as follows.

$$W=\begin{bmatrix}\sqrt{17}&0\\0&1\end{bmatrix}$$ $$V=\begin{bmatrix}\frac{1}{\sqrt{2}}&\frac{1}{\sqrt{2}}\\\frac{1}{\sqrt{2}}&\frac{-1}{\sqrt{2}}\end{bmatrix}$$


Now we can obtain the columns of U without explicitly calculating the eigenvectors of $AA^T$ as follows.

Simply use the following relation,

$$Av_i=\sigma_i u_i, \forall i$$

$$\therefore Av_1=\sigma_1 u_1$$ $$\begin{bmatrix}1&2\\2&2\\2&1\end{bmatrix} \begin{bmatrix}\frac{1}{\sqrt{2}}\\\frac{1}{\sqrt{2}}\end{bmatrix}= \sqrt{17} u_1$$

$$\therefore \begin{bmatrix}\frac{3}{\sqrt{2}}\\\frac{4}{\sqrt{2}}\\\frac{3}{\sqrt{2}}\end{bmatrix}=\sqrt{17} u_1\implies u_1=\begin{bmatrix}\frac{3}{\sqrt{34}}\\\frac{4}{\sqrt{34}}\\\frac{3}{\sqrt{34}}\end{bmatrix}$$

Also, $Av_2=\sigma_2 u_2$

$$\therefore \begin{bmatrix}1&2\\2&2\\2&1\end{bmatrix} \begin{bmatrix}\frac{1}{\sqrt{2}}\\\frac{-1}{\sqrt{2}}\end{bmatrix}= 1 u_2 \implies u_2 = \begin{bmatrix}\frac{-1}{\sqrt{2}}\\0\\\frac{1}{\sqrt{2}}\end{bmatrix}$$


$$\therefore U = \begin{bmatrix} \frac{3}{\sqrt{34}}&\frac{-1}{\sqrt{2}}\\ \frac{4}{\sqrt{34}}&0\\ \frac{3}{\sqrt{34}}&\frac{1}{\sqrt{2}}\end{bmatrix}$$

So, the SVD of A is,

$$\begin{bmatrix}1&2\\2&2\\2&1\end{bmatrix}= \begin{bmatrix} \frac{3}{\sqrt{34}}&\frac{-1}{\sqrt{2}}\\ \frac{4}{\sqrt{34}}&0\\ \frac{3}{\sqrt{34}}&\frac{1}{\sqrt{2}}\end{bmatrix} \begin{bmatrix}\sqrt{17}&0\\0&1\end{bmatrix} \begin{bmatrix}\frac{1}{\sqrt{2}}&\frac{1}{\sqrt{2}}\\\frac{1}{\sqrt{2}}&\frac{-1}{\sqrt{2}}\end{bmatrix}$$


The above is the reduced SVD. In any case if you want to obtain the full SVD, all you have to do is obtain the normalized eigenvector of $AA^T$ corresponding to the eigenvalue of $0$ and, put that as the third column of the above U matrix. Also, remember to add another row of zeros to the W matrix as its third row.

The above method can be applied to any given matrix to obtain the singular value decomposition.