Why does sample covariance matrix inflates the larger eigenvalues and shrinks the smaller eigenvalues of the covariance matrix?

112 Views Asked by At

In classification context, using the maximum likelihood, the sample covariance matrix for each class, estimates the larger eigenvalues of the covariance matrix of each class, larger and estimates the smaller eigenvalues smaller. Why is that? I am studying Regularized Discriminant Analysis written by Friedman.

From Regularized Discriminant Analysis Jerome H. Friedman, Journal of the American Statistical Association, Vol. 84, No. 405 (Mar., 1989), pp. 165-175 (original source):

It is well known that the estimates based on Equation (12) produce biased estimates of the eigenvalues; the largest ones are biased high and the smallest ones are biased toward values that are too low. This bias is most pronounced when the population eigenvalues tend toward equality, and it is correspondingly less severe when their values are highly disparate. In all cases, this phenomenon becomes more pronounced as the sample size decreases. When $N_k \le p$ the sample covariance matrix is singular with rank $\le N_k$ and the smallest $p - N_k + 1$ eigenvalues are estimated to be 0. The corresponding eigenvectors are then arbitrary, subject perhaps to orthogonality constraint