High first singular value for random matrix

Question

I have a 100 by 100 matrix of random entries. The SVD of the matrix shows that the first singular value is quite high and the rest are substantially lower. I know that this first singular value is the best rank one approximation of the matrix but why is it so much higher than the other values?

Answer 1

My guess is you are selecting the entries according to a distribution whose mean is not $0$, e.g. uniform on $\{ 0, 1 \}$ or $[0, 1]$. If the entry distribution satisfies $\mathbb{E}(X) = \mu \neq 0$ then your matrix is a mean-zero perturbation of the matrix all of whose entries are $\mu$, which has rank $1$. This predicts that you should find a singular value close to $|\mu|$, with singular vector close to $(1, 1, 1, \dots )$, which appears for quite different reasons than the others.

You can check if this explanation is correct by replacing your entry distribution $X$ with $X - \mu$ and seeing what happens.

Answer 2

Much enlightenment can be gleaned from [this paper][1]

Vivo, Pierpaolo; Majumdar, Satya N.; Bohigas, Oriol, Large deviations of the maximum eigenvalue in Wishart random matrices, J. Phys. A, Math. Theor. 40, No. 16, 4317-4337 (2007). ZBL1115.15019. [1]: https://arxiv.org/abs/cond-mat/0701371