Why do $M^\dagger M$ amd $MM^{\dagger}$ have the same first $k$ eigenvalues?

Question

In the derivation of the singular value decomposition, it is stated without proof in my notes that there is a relationship between the singular values of $M$ and the eigenvalues of $M^\dagger M$ amd $MM^{\dagger}$, namely

$$s_k(M) = \sqrt{\lambda_k(MM^{\dagger})} = \sqrt{\lambda_k(M^{\dagger}M)},$$

for $1\leq k\leq rank(M)$.

For non-square $M$, how can this be proved?

Answer 1

Let's say we have an eigenvector $u_k$ for matrix $MM^T$ with eigenvalue $\lambda_k\neq 0$: $$ \lambda_k u_k = MM^Tu_k. $$

Then consider how matrix $M^TM$ acts on vector $v_k = M^Tu_k$: $$ M^TMv_k = (M^TM)M^T u_k = M^T (MM^Tu_k)=M^T\lambda_ku_k=\lambda_k(M^Tu_k) = \lambda_kv_k. $$ So $v_k$ is an eigenvector for $M^TM$ with the same eigenvalue $\lambda_k$. Finally, since we have $\mathrm{rank}\ M$ non-zero eigenvalues, it would work for every one of them.