Estimation of the eigenvectors of a covariance matrix in the presence of misspecified eigenvalues.

107 Views Asked by At

I have the following objective function: $$f(L) = E[y'L^{\phantom{'}}D^{-1}L'y],$$ where $E[\,\cdot\,]$ is the expectation operator; $D$ is a $k$-dimensional diagonal matrix with strictly decreasing diagonal elements, $d_{1}>d_{2}>\ldots>d_{k}>0;$ $y$ is a $k \times 1$ random vector with zero mean and covariance matrix $\bar S$, and $L \in \mathcal{L}_{k},$ where $\mathcal{L}_{k}$ is the set of the $k$-dimensional orthonormal matrices (that is: $L'L = I$ where $I$ is the identity matrix). Let $$\bar S = \bar L \bar D \bar L'$$ denote the spectral decomposition of $\bar S$, assumed with strictly decreasing eigenvalues, $\bar d_{1}>\bar d_{2}>\ldots>\bar d_{k}>0.$ My conjecture is that $f(L)$ has a global minimum at $L = \bar L,$ where $f(L) = E[y'L^{\phantom{'}}D^{-1}L'y] = E[{\rm trace}(y'L^{\phantom{'}}D^{-1}L'y)]= E[{\rm trace}(L^{\phantom{'}}D^{-1}L'yy')] = {\rm trace}(L^{\phantom{'}}D^{-1}L'E[yy']) = {\rm trace}(L^{\phantom{'}}D^{-1}L'\bar S) = {\rm trace}(L^{\phantom{'}}D^{-1}L'\bar L \bar D \bar L') = {\rm trace}(D^{-1}L'\bar L \bar D \bar L'L^{\phantom{'}}) = {\rm trace}(D^{-1} \bar D) = \frac{\bar d_{1}}{d_{1}}+\cdots+\frac{\bar d_{k}}{d_{k}}.$

By similar arguments, $f(L)$ should have a global maximum at $L = \tilde L,$ where $\tilde L$ is the eigenvector matrix associated to the increasing eigenvalues of $\bar S;$ that is: the first column of $\tilde L$ is the last column of $\bar L,$ the second column of $\tilde L$ is the second-last column of $\bar L,$ and so on. If $L = \tilde L,$ I have $$f(L) = E[y'L^{\phantom{'}}D^{-1}L'y] = \frac{\bar d_{1}}{d_{k}}+\cdots+\frac{\bar d_{k}}{d_{1}}.$$

If my conjecture is true, I can estimate the eigenvectors of $\bar S$ by minimizing/maximizing the sample counterpart of $f(L)$, that is: $$g(L) = \frac{1}{n}\sum^{n}_{i=1}y'_{(i)}L^{\phantom{\prime}}D^{-1}L'y_{(i)},$$ where $y_{(1)},\ldots,y_{(n)}$ is a random sample from the distribution of $y$. This estimator can be seen as an eigenvector estimator of $\bar S$ which is robust to misspecifications of the eigenvalues ($D \neq \bar D$). Unfortunately, for reasons that are too long to explain here, I cannot simply estimate the eigenvectors of $\bar S$ by the eigenvectors of the sample covariance matrix.

Unfortunately, I cannot prove my conjecture on the minima and the maxima of $f(L)$.

Can anyone help me to say if my conjecture is true? And, if this is the case, can anyone help me to prove it?

Thanks in advance.