I use the Random Matrix Theory to filter out the information from the correlation matrix that is associated with noise - Marcenko Pastur band. That is straight forward.
Then I follow Rosenow, Bernd, et al. "Portfolio optimization and the random magnet problem." EPL (Europhysics Letters) 59.4 (2002): 500. The procedure described in the paper
It says how to reconstruct the correlation matrix with filtered eigenvalues in the original basis.
My logic: I need to diagonalise a correlation matrix C, which is symmetric, the result is a sparse matrix with eigenvalues on the diagonal. The transformation matrix Q has columns of the respective eigenvectors.
computation $D=diag(t(Q)\%*\%empCor\%*\%Q)$ yields indeed the eigenvalues, where $t()$ is transpose. So that's ok.
As the paper says I denoise the eigenvalues by setting to zero all lower than the Marcenko-Pastur band and sort the diagonal in ascending order. I also sort the respective eigenvectors (although they dont specifically say) with it.
Now I understand I need to transform the to transform the new D_hat to basis of C: I would do C_hat= Q%%D_hat%%t(Q), but it always yields one negative eigenvalue of the new matrix, and hence the new correlation matrix is no longer positive(semi) definite. I cannot figure out why it happens, because the paper does not specify there needs to be any correction done.
I figured the problem out, if anyone ever needs it. It is based on Brian Rowe's tawny package in R
1) Work with correlation, not covariance, matrix!
2) Get eigenvalues and eigenvectors (save those)
3) Replace all eigenvalues below upper Marcenko-Pastur band with their average
4) create a diagonal matrix with the new eigenvalues. Change into old basis
$C_{hat}=Q*D_{hat}*Q^{T}$
5) Apply scaling correction $DiagM=diag(C_{hat})*o*I$ where $*o*$ is the outer product and $I$ is the unit matrix. 6) Resulting denoised matrix is
$C_{hat}=\frac{C_{hat}}{\sqrt{DiagM \times DiagM^{T}}}$, where $\times$ is the elementwise product.