Singular Value Decomposition of a rank 1 matrix

Question

I understand that when I do SVD of a rank 1 matrix constructed as $xx^T=U\Sigma U^T$, where U=$\frac{x}{\Vert{x\Vert}}$. But when I calculate $UU^T$ I do not get the identity matrix. What am I doing wrong, I thought that $UU^T=I$ is a given with SVD?

Answer 1

This is something that depends on the convention you are using for your singular value decomposition.

Let $A$ be a $m \times n$ matrix with rank $r$, so that $A$ has $r$ nonzero singular values. One convention for the SVD is to write $A=U \Sigma V^T$ where $\Sigma$ is an $r \times r$ matrix diagonal matrix whose diagonal entries are the nonzero singular values of $A$. In this case $U$ is an $m \times r$ matrix that has orthonormal columns and $V$ is an $n \times r$ matrix with orthonormal columns. Since $U$ has orthonormal columns we have $U^T U=I_r$, as is the case in your SVD. (Similarly $V^T V= I_r$). However, we can't have $U U^T=I_m$, unless $r=m$. As a side note, if $r=m$ then we necessarily do have $U U^T=I_m$. The reason we can't have $U U^T=I_m$ if $r \neq m$ is, if $r \neq m$ then we must have $r < m$. Since $U$ has orthonormal columns, the row rank of $U^T$ is $r$. Therefore, the rank of $U^T$, hence of $U U^T$ is $r$.

A second convention for the SVD is to write $A= \hat{U} \hat{\Sigma} \hat{V}^T$ where $\hat{U}$ is $m \times m$ and $\hat{\Sigma}$ is $m \times n$ and $\hat{V}$ is $n \times n$. In this case the $\hat{\Sigma}$ is still a diagonal matrix, but now we include the zero singular values (counting multiplicity) on the diagaonal of $\hat{\Sigma}$. Our matrix $\hat{U}$ still has orthonormal columns, so we still have $\hat{U}^T \hat{U}=I_m$. However, since $\hat{U}$ is now square, $\hat{U}^T \hat{U}=I_m$ implies that $\hat{U}^T$ is the inverse of $\hat{U}$. That is, $\hat{U}$ is an orthogonal matrix, hence $\hat{U} \hat{U}^T=I_m$.

One can check that if the singular values are arranged in decreasing order on the diagonal in both cases, then (ignoring some minor uniquness issues which occur with repeated singular values) we have $$ \hat{U}=(U \ \ N) $$ where $U$ is the matrix from the first case, and the columns of $N$ form an orthonormal basis for the orthogonal complement of the range of $A$.

The short version is that you haven't done anything wrong. You have just slightly confused properties of two different conventions for the SVD.