Why does scaling a column of $U$ by $-1$ make the SVD of a matrix incorrect?

50 Views Asked by At

Consider the SVD of a matrix $A = U\Sigma V^T$ where $U = [u_1 \; u_2 \; u_3 \; ... \; u_m]$.

If I scale a column of $U$ by $-1$ to, for example, $U^{\prime} = [-u_1 \; u_2 \; u_3 \; ... \; u_m]$, $U^{\prime}$ remains orthogonal and $-u_1$ is still a unit eigenvector of $AA^T$ with the associated eigenvalue $\sigma_1^2$.

However, if I now set $A^{\prime} = U^{\prime}\Sigma V^T$, I get $A \neq A^{\prime}$.

Why does $U^{\prime}\Sigma V^T$ not give me a valid SVD of $A$?

Example:

$$\begin{bmatrix}-1/3 & \frac{2\sqrt{5}}{5} & -\frac{2\sqrt{5}}{15}\\ 2/3 & \frac{\sqrt{5}}{5} & \frac{4\sqrt{5}}{15}\\ -2/3 & 0 & \frac{\sqrt{5}}{3}\end{bmatrix} \begin{bmatrix}3\sqrt{2} & 0 \\ 0 & 0\\ 0 & 0\end{bmatrix}\begin{bmatrix}-\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}}\end{bmatrix} \neq \begin{bmatrix}1/3 & \frac{2\sqrt{5}}{5} & -\frac{2\sqrt{5}}{15}\\ -2/3 & \frac{\sqrt{5}}{5} & \frac{4\sqrt{5}}{15}\\ 2/3 & 0 & \frac{\sqrt{5}}{3}\end{bmatrix} \begin{bmatrix}3\sqrt{2} & 0 \\ 0 & 0\\ 0 & 0\end{bmatrix}\begin{bmatrix}-\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}}\end{bmatrix}$$

2

There are 2 best solutions below

2
On BEST ANSWER

$U$ and $V$ cannot be created independently of each other. You cannot simply use any $U$ that diagonalizes $AA^T$ and any $V$ that diagonalizes $A^T\!A.$ Calculating an SVD is more than that. $U$ and $V$ must be "compatible".

In your case, you can also scale the first column of $V$ (the first row of $V^T$) to get correct results, again.

However, this is not the only thing that can go wrong. If $AA^T$ and $A^T\!A$ have an eigenvalue $\sigma^2$ with multiplicity greater than $1$, you cannot simply use any set of orthogonal eigenvectors that span the eigenspace of $AA^T$ associated with $\sigma^2$ to fill your $U$ and any set of orthogonal eigenvectors that span the eigenspace of $A^T\!A$ associated with $\sigma^2$ to fill your $V.$ Those basis vectors must also be "compatible".

2
On

\begin{align} A= \sum_{i=1}^m \sigma_i u_i v_i^\top = \sum_{i=1}^m \sigma_i (-u_i) (-v_i^\top) \end{align}

You need to also change the sign of the associated right singular vector $v_i$, in your case $v_1$.