SVD Singular Value Issue

Question

I am having an issue with a SVD problem and would like advice. The matrices given are

$$ A= \left[ \begin{matrix} 1 & 2\\ 3 & 6 \\ \end{matrix}\right]\ \ \ A^TA= \left[ \begin{matrix} 10 & 20\\ 20 & 40 \\ \end{matrix}\right]\ \ \ AA^T= \left[ \begin{matrix} 5 & 15\\ 15 & 45 \\ \end{matrix}\right] $$

I initially determined my $λ$ values to be $λ_1=50$ and $λ_2=0$ by computing the following:

$$ det⁡(A^TA-λI)=0\\ \begin{vmatrix} 10-λ & 20\\ 20 & 40-λ \\ \end{vmatrix}\\ (10-λ)(40-λ)-20^2=0\\ 400-50λ+λ^2-400=0\\ λ^2-50λ=0\\ λ(λ-50)=0 $$

This in turn means that my singular values will be $\sigma_1=\sqrt {50}$ and $\sigma_2=0$, and this is where I run into my issue. When I am trying to solve for the orthonormal set with $u_1$ and $u_2$, I was told to use the equation $u_n=\frac{Av_n}{\sigma_n}$. I cannot compute $u_2$ because that would make the denominator $0$, and the fraction would become undefined.

I discussed with my professor and they said I should try to calculate $u_2$ using the eigenvectors from $AA^T$, but $AA^T$ and $A^TA$ have the same eigenvalues. I thought maybe they meant to compute the $u$ for the $\sigma=\sqrt {50}$, but this is not the same answer as the one the book gives. The answers for this and Wolfram Alpha show that the SVD should be

$$ \left[ \begin{matrix} 1 & 2\\ 3 & 6 \\ \end{matrix}\right]= \frac{1}{\sqrt{10}} \left[ \begin{matrix} 1 & -3\\ 3 & 1 \\ \end{matrix}\right] \left[ \begin{matrix} \sqrt{50} & 0\\ 0 & 0 \\ \end{matrix}\right] \frac{1}{\sqrt{5}} \left[ \begin{matrix} 1 & -2\\ 2 & -1 \\ \end{matrix}\right] $$

I have gotten that $u_1=\frac{1}{\sqrt{10}} \left[ \begin{matrix} 1 \\ 3 \\ \end{matrix}\right]$, but I do not know how to get $u_2=\frac{1}{\sqrt{10}} \left[ \begin{matrix} -3 \\ 1 \\ \end{matrix}\right]$. Any help would be appreciated.

Answer 1

Your professor was right.

Have you tried the definition of an eigenvector?

$$(AA^T)v=\lambda v$$

$$ \left[ \begin{matrix} 5 & 15\\ 15 & 45 \\ \end{matrix}\right] \left[ \begin{matrix} v_1\\ v_2\\ \end{matrix}\right] =\left[ \begin{matrix} 0\\ 0\\ \end{matrix}\right] $$

$$ \begin{align} & 5v_1+15v_2=0\\ & 15v_1+45v_2=3(5v_1+15v_2)=0 \end{align} $$

So you have one degree of freedom to choose $v_2 = 1/\sqrt(10)$ and obtain $v_1 = -15/5 \, v_2 = -3/\sqrt(10)$.

Answer 2

Note that another way to write the singular value decomposition is to write it as the sum of rank one matrices: $$A = U D V^T = \sum_{k = 1}^n \sigma_i \vec{u}_i \vec{v}_i^T$$ where $\vec{u}_i$ is the $i$th column of $U$ and $\vec{v}_i$ is the $i$th column of $V$. What does this tell us? It tells us that whenever $\sigma_i = 0$, the resulting left and right singular vectors don't really matter. So you have two choices:

1. If $A$ is $m \times n$ (here $m = n = 2$) You can complete $\vec{u}_1$ to an orthonormal basis $\{\vec{u}_1, \vec{u}_2, \cdots, \vec{u}_m\}$ of $\mathbb{R}^m$ and complete $\vec{v}_1$ to an orthonormal basis $\{\vec{v}_1, \vec{v}_2, \cdots, \vec{v}_m\}$ of $\mathbb{R}^m$ of $\mathbb{R}^n$ (this can be done with Gram-Schmidt). As a result, $D$ will be an $m \times n$ diagonal matrix. That's what the solutions have done. Then we may write $$A = \begin{bmatrix} \vec{u}_1 & \cdots & \vec{u}_m \end{bmatrix} \cdot \text{diag}_{m \times n}(\sigma_1, \sigma_2, \cdots, \sigma_{\min(m,n)}) \cdot \begin{bmatrix} \vec{v}_1 & \cdots & \vec{v}_n \end{bmatrix}^T$$

2. Calculate the reduced singular value decomposition. This involves only worrying about the nonzero singular values, so we won't need to complete any set of singular vectors to a basis. Here $D$ will become an $r \times r$ diagonal matrix, where $r$ is the number of nonzero singular values ($r$ is also the rank of $A$). $U$ will be $n \times r$, and $V$ will be $m \times r$. In this case, we know $r = 1$ and you've calculated the corresponding left and right singular vectors of $\sigma_1 = \sqrt {50}$. So you can write $$A = \frac{1}{\sqrt{10}}\begin{bmatrix} 1 \\ 3\end{bmatrix} \begin{bmatrix} \sqrt{50} \end{bmatrix} \left(\frac{1}{\sqrt 5} \begin{bmatrix} 1 & -2\end{bmatrix}\right)$$ Here $U$ and $V$ are partially orthogonal, in the sense that $U^T U = I_r = V^T V$ but $UU^T \neq I_n$ and $V V^T \neq I_m$.