Singular value decomposition works only for certain orthonormal eigenvectors, not all?

Question

I'm trying to find the SVD of the following matrix: $$A= \begin{pmatrix} 1 & 1 \\ 2 & -2 \\ 2 & 2 \\ \end{pmatrix} $$

I found the eigenvalues and vectors for $A'A$: $$ \begin{array}{cc} \lambda_1=10 & \lambda_2=8 \\ e_1'=(1,1) & e_2'=(-1,1). \\ \end{array} $$

I find the eigenvalues and vectors for $AA'$: $$ \begin{array}{ccc} \lambda_1=10 & \lambda_2=8 & \lambda_3=0 \\ e_1=(1,0,2) & e_2=(0,1,0) & e_3=(-2,0,1), \\ \end{array}$$

and so my SVD should be: $$\left( \begin{array}{ccc} \frac{1}{\sqrt{5}} & 0 & -\frac{2}{\sqrt{5}} \\ 0 & 1 & 0 \\ \frac{2}{\sqrt{5}} & 0 & \frac{1}{\sqrt{5}} \\ \end{array} \right).\left( \begin{array}{cc} \sqrt{10} & 0 \\ 0 & \sqrt{8} \\ 0 & 0 \\ \end{array} \right).\left( \begin{array}{cc} \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\ -\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\ \end{array} \right)$$

However, this gives $$\left( \begin{array}{cc} 1 & 1 \\ -2 & 2 \\ 2 & 2 \\ \end{array} \right)$$ instead of $A$.

To get $A$ I need to decompose in the following way $$\left( \begin{array}{ccc} \frac{1}{\sqrt{5}} & 0 & -\frac{2}{\sqrt{5}} \\ 0 & 1 & 0 \\ \frac{2}{\sqrt{5}} & 0 & \frac{1}{\sqrt{5}} \\ \end{array} \right).\left( \begin{array}{cc} \sqrt{10} & 0 \\ 0 & \sqrt{8} \\ 0 & 0 \\ \end{array} \right).\left( \begin{array}{cc} \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{2}} \\ \end{array} \right)$$

This last decomposition is as if I had multiplied the first $e_2$ by $-1$.

These eigenvalues and vectors were derived from Mathematica, just to be sure I was using the correct elements.

Any help in explaining why my first decomposition doesn't work would be appreciated.

Edit: The book I'm using doesn't tell which of the orthonormal eigenvectors I have to use. For each eigenvalue, I have two orthonormal eigenvectors, $e_i$ and $-e_i$.

Answer 1

You need to match the left singular vectors to the right ones, or vice versa. E.g. after you have computed $e_1'$ and $e_2'$, you could get the two corresponding left singular vectors as $e_1=Ae_1'/\|Ae_1'\|=\frac1{\sqrt{5}}(1,0,2)^T$ and $e_2=Ae_2'/\|Ae_2'\|=(0,\color{red}{-1},0)^T$ (note: the sign of $e_2$ here is different from yours). The remaining left singular vector can be any unit vector orthogonal to the previous two singular vectors (in this case, it must be $\pm e_1\times e_2$, where the sign is unimportant). Then \begin{align*} A&= \pmatrix{e_1&e_2&\pm e_1\times e_2} \pmatrix{\frac{\|Ae_1'\|}{\|e_1'\|}&0\\ 0&\frac{\|Ae_2'\|}{\|e_2'\|}\\ 0&0} \pmatrix{\frac{(e_1')^T}{\|e_1'\|}\\ \frac{(e_2')^T}{\|e_2'\|}}\\ &=\pmatrix{\frac{1}{\sqrt{5}}&0&\frac{\pm2}{\sqrt{5}}\\ 0&-1&0\\ \frac{2}{\sqrt{5}}&0&\frac{\mp1}{\sqrt{5}}} \pmatrix{\sqrt{10}&0\\ 0&\sqrt{8}\\ 0&0} \pmatrix{\frac{1}{\sqrt{2}}&\frac{1}{\sqrt{2}}\\ \frac{-1}{\sqrt{2}}&\frac{1}{\sqrt{2}}}. \end{align*}

Answer 2

The answer is yes. Since your $m\times n$ matrix $A$ is such that $m\geq n$, you first compute $\sigma_i$s and $v_i$s and than get the $u_i$s via relation

$$u_i = \frac{1}{\sigma_i}Av_i\text{.}$$ This follows from the proof of "SVD theorem": There exist unitary matrices $U$ and $V$ with the columns $u_i$ and $v_i$ (and matrix $\Sigma$ ...) for which $A = U\Sigma V^H$.

If you choose, for example $-u_i$s from the upper relation (which are still eigenvectors of $AA'$), you would get $U\Sigma V^H = -A$.

Answer 3

I'm posting an answer just to complete the previous answers.

Let $A=UDV'$. Then, if we have dim of $U$ greater than $V$ we can use the formula $v_i=\frac{1}{\sigma_i} A'u_i$, for given $U$.

However, if we have dim of $U$ smaller than $V$, we can use the formula $u_i=\frac{1}{\sigma_i}Av_i$, for given $V$.