U matrix in Singular value decompositon.

Question

I know that the Singular Value Decomposition of a matrix $X$ is given by:

$X = U\Sigma V^T$, where $U$ and $V$ matrices are column orthonormal and $\Sigma$ is a diagonal square matrix containing singular values.

Does $U$ contain the eigenvectors of $XX^T$? Why or why not?

Answer 1

The SVD is formed from the covariance matrices $X^{T}X$ and $XX^{T}$. you can show this.

$$ XX^{T} = (U \Sigma V^{T}) (U \Sigma V^{T})^{T} \tag{1} $$ $$ XX^{T} = U \Sigma V^{T} V \Sigma^{T} U^{T} \tag{2} $$

we know that $VV^{T} = V^{T}V = I$ $$ XX^{T} = U \Sigma \Sigma^{T} U^{T} \tag{3} $$

also $ \Sigma^{T} \Sigma = \Sigma \Sigma^{T} = \Lambda$ $$ XX^{T} = U \Lambda U^{T} \tag{4} $$ $U $ are left singular vectors. Note that they are eigenvectors of the covariance matrix $XX^{T}$ similarly

$$ X^{T}X = (U \Sigma V^{T})^{T} (U \Sigma V^{T}) \tag{5} $$ $$ X^{T}X = V \Sigma^{T} U^{T} U \Sigma V^{T} \tag{6} $$ $ U^{T}U = UU^{T} = I $ $$ X^{T}X = V \Sigma^{T} \Sigma V^{T} \tag{7} $$ $$ X^{T}X = V \Lambda V^{T} \tag{8} $$

$V $ are right singular vectors. They are eigenvectors $X^{T}X$ like above

Answer 2

It suffices to evaluate $XX^T=U\Sigma V^TV\Sigma U^T=U\Sigma^2U^T=U\Sigma^2U^{-1}.$

Then the Eigenvalues of $XX^T$ are the squares of the singular values and its Eigenvectors indeed form $U$.