I'm trying to get intuition about SVD through explaining it thoroughly and would like some feedback on my progress so far
$A$ is an $n\times m$ matrix where $m\ge n$
- $A^TA$ is an $m \times m$-matrix and has eigenvalues $\lambda_k\ge 0$ and eigenvectors $\vec{v}_k$, $k\in \{1,2,...,m\}$. $n$ of these eigenvalues are greater than zero and the rest are zero.
- $AA^T$ is an $n \times n$-matrix and has the same eigenvalues $\lambda_k\gt 0$ and eigenvectors $A\vec{v}_k$, $k\in \{1,2,...,n\}$ where $A\vec{v}_k$ is not the null-vector.
This comes from the fact that $$A^TA\vec{v}_k=\lambda_k\vec{v}_k\Leftrightarrow AA^T(A\vec{v}_k)=\lambda_k(A\vec{v}_k)$$ $$\lambda_k||\vec{v}_k||^2=A^TA\vec{v}_k\cdot \vec{v}_k=(A\vec{v}_k)\cdot(A\vec{v}_k)=||A\vec{v}_k||^2$$
If $A^TA$ has an eigenvalue $\lambda_k=0$, then the corresponding map of that eigenvector $A\vec{v}_k$ is the null vector. Since $AA^T$ is symmetrical and must have a full number of eigenvectors, $A^TA$ must have at least $n-number$ of eigenvalues $\lambda_k>0$
If $A^TA$ has an eigenvalue and corresponding eigenvector $\lambda_k>0,\vec{v}_k$, and $A\vec{v}_k$ is not an eigenvector of $AA^T$, then the equation $AA^T(A\vec{v}_k)=\lambda_k(A\vec{v}_k)$ has no solutions. Hence $A^TA$ has $n$-number of eigenvalues $\lambda_k>0$ and the rest are $0$.
- $A^TA=VD_VV^T$ and $AA^T=UD_UU^T$ where $U,V$ are orthogonal matrices with eigenvectors as column vectors. $D_V,D_U$ are diagonal matrices with eigenvalues as diagonal elements. This comes from the fact that they are symmetrical matrices and thus diagonalisable.
$$A\vec{x}=A_VV^T\vec{x}\Leftrightarrow A_V=AV=(A\vec{v}_1\ ...\ A\vec{v}_m)$$, where $\vec{v}_k,\ k\in \{1...m\}$ are the eigenvectors of $A^TA$ with the first $n$ eigenvalues $\lambda_k>0$ and the rest being zero. Since $V$ is an orthogonal matrix, $||\vec{v}_k||=1$ $$A_V=(\frac{\sqrt{\lambda_1}}{||A\vec{v}_1||}A\vec{v}_1\ ...\ \frac{\sqrt{\lambda_n}}{||A\vec{v}_n||}A\vec{v}_n\ ...\ \vec{0})$$ $$U=(\frac{A\vec{v}_1}{||A\vec{v}_1||}\ ...\ \frac{A\vec{v}_n}{||A\vec{v}_n||})$$ $\Sigma$ is a $n\times m$-diagonal matrix with $\lambda_k,\ k\in \{1,...,n\}$ as diagonal elements and the $m-n$ last column vectors being null-vectors. $$A_V=U\Sigma\Rightarrow A=U\Sigma V^T$$
This makes some sense to me, unless I have made some misstake. $A$ maps the orthonormal eigenvectors of $A^TA$ on to the orthonormal eigenvectors of $AA^T$, scaled by the root of their shared eigenvalues. The SVD is expressing any vector in $A^TA$:s eigenvector coordinates, scaling by the root of the eigenvalues and then expressing the resulting vector from the base of $AA^T$:s eigenvectors to the standard base.
$A$ is an $n\times m$ matrix where $n\gt m$
- $AA^T$ is an $n \times n$-matrix and has eigenvalues $\lambda_k\ge 0$ and eigenvectors $\vec{v}_k$, $k\in \{1,2,...,n\}$. $m$ of these eigenvalues are greater than zero and the rest are zero.
- $A^TA$ is an $m \times m$-matrix and has the same eigenvalues $\lambda_k\gt 0$ and eigenvectors $A^T\vec{v}_k$, $k\in \{1,2,...,m\}$ where $A^T\vec{v}_k$ is not the null-vector.
This comes from the fact that $$AA^T\vec{v}_k=\lambda_k\vec{v}_k\Leftrightarrow A^TA(A^T\vec{v}_k)=\lambda_k(A^T\vec{v}_k)$$ $$\lambda_k||\vec{v}_k||^2=AA^T\vec{v}_k\cdot \vec{v}_k=(A^T\vec{v}_k)\cdot(A^T\vec{v}_k)=||A^T\vec{v}_k||^2$$
Through the same process as above I come to the statement $A^T=U\Sigma V^T$ and then $A=V\Sigma^TU^T$