A question in SVD

Question

if $A\in \mathbb{R}^{m \times n}$, then SVD of A is $U\Sigma V^T$. I've seen different version of SVD. In the first one, $U\in \mathbb{R}^{m \times m}$, $\Sigma \in \mathbb{R}^{m\times n}$, and $V \in \mathbb{R}^{n\times n}$. In the second one $U\in \mathbb{R}^{m \times r}$, $\Sigma \in \mathbb{R}^{r\times r}$, and $V \in \mathbb{R}^{r\times n}$, where $r$ is the rank of $A$. Is there difference in these two forms or does one of them have an advantage over the other one? For instance when $U$ and $V$ are square matrices, then they are invertible, i.e, $U^TU=UU^T=I$. but when $U$ is not square only $U^TU=I$ is true.

Second question. if $A=U\Sigma V^T$, I've seen that $U^TU=I$. Can we deduce that $UU^T=I$, too?

Answer 1

First question

It follows from the definition of the SVD. The first one

$$ A_{m \times n} = U_{m \times m} \Sigma_{m \times n} V_{n \times n}^{T} \tag{1} $$

is the full SVD

the second one

$$ A_{m \times n} = U_{m \times r} \Sigma_{r \times r} V_{r \times n}^{T} \tag{2} $$

is the truncated SVD or reduced SVD where $A$ has rank $r$

visually we see the difference

Second question

It follows from the definition of the SVD. $U$ is orthogonal, which means that

$$ U^{T}U = UU^{T} = I \tag{3} $$

Note

In the SVD

$$ \sigma_{1} \geq \sigma_{2} \geq \cdots \sigma_{r} \geq 0 \tag{4} $$

if you multiply

$$ U_{m \times m} \cdot \Sigma_{m \times m} V_{m \times n}^{T}\tag{5} $$

and there are only $r$ singular values then it is $0$ after that. A diagonal matrix simply multiplies across. So we only care about

$$ U_{m \times r}\Sigma_{r \times r} V_{r \times n}^{T} \tag{6} $$

Answer 2

Basically these two forms are equivalent. The second form can be deduced from the first form, whose elements are those of the second form and lots of zero, So when applied to data storage the second form take advantage, because you can ignore plenty of zeros and thus deal with a lower dimension. Another thing is, if U is not a square matrix then the conclusion is not true. Take (0 1) as an example. Also you may consider the rank of UU' and U'U, one of whom can't be full-rank. But there's indeed a conclusion that they share the same nonzero egienvalues, including their dimension!