Relation between SVD and EVD

Question

Given SVD decomposition $ A = U \Sigma V^T $ (where $U$ and $V$ are orthonormal and $ \Sigma $ is a diagonal matrix), I wish to prove that $ AA^T=U\Sigma \Sigma ^TU^T $ is the EVD decomposition of $ AA^T $ (same goes for $ A^TA=V^T \Sigma ^T \Sigma V $).

It's easy to see that indeed $ AA^T=U\Sigma \Sigma ^TU^T $. But I don't understand why the values on $ \Sigma \Sigma ^T $'s diagonal are $ AA^T $'s eigenvalues.

Answer 1

You forgot a transpose: $A A^T = U \Sigma \Sigma^T U^T$. With that said, recall that by assumption $U^T = U^{-1}$. Now you can see by direct substitution that the columns of $U$ are the eigenvectors of $A A^T$ and that the eigenvalues are the diagonal entries of $\Sigma \Sigma^T$. From a more algebraic point of view, if you can similarity-transform a (square) matrix into diagonal form, then the diagonal entries of that diagonal matrix must be its eigenvalues.

The situation is slightly different for the "economy" SVD, but still essentially the same.

Answer 2

The product matrices $\mathbf{A}^{*} \mathbf{A}$ and $\mathbf{A} \mathbf{A}^{*}$ are symmetric. Symmetric matrices can be diagonalized by unitary matrices. The column vectors of the unitary matrices are the eigenvectors. The diagonal matrix is composed of the eigenvalues.

Answer 3

We can clearly show that by matrix transformations:

$$A = U\Sigma V^{T}$$ Using $(AB)^T = B^TA^T$: $$AA^T = U\Sigma V^TV\Sigma ^TU^T$$ Using $V^TV = I$ for unitary matrix $V$, and $\Sigma $ being diagonal: $$AA^T = U\Sigma ^2 U^T$$ Then multiplied by $U$ from the right: $$AA^TU = U\Sigma ^2$$ Which is exactly the definition of $U$ as eigenvectors of $AA^T$ and $\Sigma ^2$ as its eigenvalues.