Eigenvalues of $\mathbf{A}^\top\mathbf{A}$

Question

1. Background

Given a matrix $\mathbf{A}\in\mathbb{R}^{M\times N}$ with $M\le N$ and $\mathbf{A} \mathbf{A}^\top =\mathbf{I}_M$, I study some properties of $\mathbf{A}^\top \mathbf{A}$. Especially, I found that it always have $M$ eigenvalues close to $1$.

In my experiments, I write the following Python code based on PyTorch framework:

import torch

X = torch.randn(10, 10)
U, S, V = torch.linalg.svd(X)
A = (U @ V)[:5]  # create an orthonormal matrix
print(torch.linalg.eigvals(A.t() @ A))

One output is:

tensor([3.0804e-08+0.j, 5.8120e-08+0.j, 2.1118e-10+0.j, 1.0000e+00+0.j, 1.0000e+00+0.j,
        1.0000e+00+0.j, 1.0000e+00+0.j, 1.0000e+00+0.j, 4.8640e-08+0.j, 7.6686e-09+0.j])

We can see that there are $M=5$ eigenvalues close to $1$.

---

2. Question

Does always $M$ eigenvalues of $\mathbf{A}^\top \mathbf{A}$ be $1$? How to show that?

Answer 1

Given $A\in\mathbb{R}^{m\times n}$, where $m<n$ and $AA^\top=I_m$, the $n$-by-$n$ matrix $A^\top A$ is of rank $m$, and thus it has $n-m$ zero eigenvalues. At the same time, if we write $$A = \begin{bmatrix}a_1^\top \\ \vdots \\ a_m^\top\end{bmatrix},\ A^\top = \begin{bmatrix}a_1 & \cdots & a_m\end{bmatrix},$$ where $a_i\in\mathbb{R}^n$, then $a_i^\top a_i=1$ and $a_ia_j=0$ for $i\ne j$.

Then $a_i^\top A^\top A a_i = 1$ for all $i=1,\ldots,m$. Thus, the matrix $A^\top A$ has $m$ eigenvalues equal to 1.