I have recently started reading about SVD. If factorization of a matrix $A$ is required, we calculate the eigenvectors of $AA^T$ and $A^TA$ and they become the column vectors of $U$ and $V$ matrices correspondingly. The $\Sigma$ matrix is filled with square roots of the eigenvalues. Now we find that the eigenvalues of $AA^T$ and $A^TA$ are the same. We also find that the obtained eigenvectors are orthogonal to each other.
Question 1:
Why are the eigenvalues of $AA^T$ and $A^TA$ the same?
Question 2:
Are eigenvectors for a matrix always orthogonal? If yes, why? If not, why are they orthogonal here?
PS - I'm new to Lin-Al, so it would be really helpful if the explanations are intuitive.
(1) Let $A$, $B$ any two matrices, and $\lambda \ne 0$ an eigenvalue of $AB$, with eigenvector $x$, say. Then $ABx = \lambda x$, and $Bx \ne 0$ (as $\lambda \ne 0$). We have $BA(Bx) = B(ABx) = \lambda Bx$, so $Bx$ is an eigenvector for $BA$ with eigenvalue $\lambda$, that is, if we denote by $\sigma(AB)$ the set of eigenvalues of $AB$: $$ \sigma(AB) -\{0\} \subseteq \sigma(BA) $$ and by symmetry of our argument $$ \sigma(AB) - \{0\} = \sigma(BA) - \{0\} $$ Note that for $0$ this need not hold: For $A = B^t = (0 \, 1)$, we have $AB = 1$, so $AB$ does not have $0$ as an eigenvalue, but $BA = \binom{0\,0}{0\, 1}$ has. So in your case, the non-zero eigenvalues of $AA^t$ and $A^t A$ are the same.
(2) No, eigenvectors of general matrices needn't be orthogonal, but eigenvectors to different eigenvalues of real, symmetric matrices must. So let $B = B^t$ be a real, symmetric matrix, $Bx = \lambda x$, $By = \mu y$, with $\lambda, \mu \in \mathbb R$ eigenvalues and $x,y \ne 0$ eigenvectors of $B$. Then we have \begin{align*} \def\sp#1{\left<#1\right>}\lambda\sp{x,y} &= \sp{\lambda x,y} \\ &= \sp{Bx,y}\\ &= \sp{x,B^ty}\\ &= \sp{x,By}\\ &= \mu\sp{x,y}\\ \iff (\lambda-\mu)\sp{x,y} &= 0 \end{align*} As $\lambda \ne \mu$, we have $\sp{x,y} = 0$.