Eigenvalues and Eigenvectors of a singular Covariance matrix

Question

I am working on a research in which my data matrix $\bf X$ has dimension of $N\times P$ where $P>>>>N$.ie. its a small sample size problem. I need to compute the covariance of $\bf X$, which is denoted by $\bf C$ with dimension $P\times P$. Is it mathematically possible to do the Eigen-value decomposition of $\bf C$ because I know that $\bf C$ is not stable matrix as its Eigen value have very high variance? Also, I need to reduce the dimension of $\bf X$ using the Eigen-vectors of $\bf C$, would it be correct if I did it using Eigen vectors of $\bf C$ without regularizing it? To add to the point, based on some preliminary calculation the covariance matrix $\bf C$ has only one signifiant eigen values , rest are all very small (almost close to zero).

Answer 1

The following paper should have answers to your question:

"Piecewise polynomial solutions to linear inverse problems"
 Authors: Per Christian Hansen, Klaus Mosegaard

The implementation of the algorithm is available at:

http://www.imm.dtu.dk/~pcha/Regutools/pptsvd.m