Singular vector of random Gaussian matrix

Question

Suppose $\Omega$ is a Gaussian matrix with entries distributed i.i.d. according to normal distribution $\mathcal{N}(0,1)$. Let $U \Sigma V^{\mathsf T}$ be its singular value decomposition. What would be the distribution of the column (or row) vectors of $U$ and $V$? Would it be a Gaussian or anything closely related?

Answer 1

Suppose the entries in a random matrix $\Omega\in\mathbb R^{n\times n}$ are i.i.d. $N(0,1)$. Then for any orthogonal matrices $G,H\in\mathbb R^{n\times n}$, the entries in $G\Omega H^T$ are also i.i.d. normal. Maybe the easiest way to see that is is this: let $\operatorname{vec}(M)$ be an $n^2\times 1$ vector in which the entries in $M\in\mathbb R^{n\times n}$ are stacked up into one column: the first column of $M$ followed by the second, and so on. Then $\operatorname{vec}(G\Omega H^T) = (G\otimes H)\operatorname{vec}(\Omega)$, where $G\otimes H$ is a Kronecker product, and then observe that $G\otimes H$ is also an orthogonal matrix. When you multiply a random vector with i.i.d. $N(0,1)$ entries by an orthogonal matrix, you get another random vector with i.i.d. $N(0,1)$ entries. That is because $\operatorname{var}(A\Omega) = A\Big(\operatorname{var}(\Omega)\Big) A^T$, so you have the variance $AIA^T = I$.

So this gives us an argument from spherical symmetry: the distribution of the singular vectors of $\Omega$ must be the same as the distribution of the singular vectors of $G\Omega H^T$; hence the distribution is invariant under rotations. Consequently the distibution of each singular vector is uniform on the sphere of radius $1$ centered at $0$ in $\mathbb R^n$.

The distribution of the singular values is a harder problem. The distribution of the singular values is of interest in statistics: Suppose you suspect that the matrix you're looking at is a low-rank matrix plus random noise, so that only the noise accounts for the data matrix having full rank. Then you might expect some large singular values that would have been nonzero even if there had been no noise, plus some small singular values that are nonzero only because of the noise. So the question is: when do the singular values of the data matrix differ significantly from those coming from your matrix $\Omega$?