Bounding the Eigenvalues of a Scaled Covariance Matrix

721 Views Asked by At

Let $X \in \mathbb{R}^{n\times p}$ be a data matrix where $n \geq p$.

Say that I know that the covariance matrix $$X^T X$$ has $n$ real eigenvalues $\lambda_1,\ldots,\lambda_n$.

Is it possible to produce a lower bound and/or upper bound on the eigenvalues of a scaled covariance matrix:

$$X^T D X$$

Here $D$ is a $n\times n$ diagonal "scaling" matrix with $d_1,\ldots,d_n$ on the diagonal. The elements $d_1,\ldots,d_n$ are real, positive, and bounded in the interval $[d_{min},d_{max}]$ where $0 < d_{min} < d_{max} < 1$.

1

There are 1 best solutions below

2
On BEST ANSWER

Upper Bound:

Let $\rho$ denote the maximal eigenvalue of $X^TX$. Let $\|A\|$ denote the spectral norm of the matrix $A$, given by $\|A\| = \lambda_{max}(\sqrt{A^TA})$. We then have $$ \|X^TDX\| \leq \|X^T\|\cdot \|D\| \cdot \|X\| $$ On the other hand, we have $$ \|X\| = \sqrt{\rho}\\ \|X^T\| = \lambda_{max}(\sqrt{XX^T}) = \lambda_{max}(\sqrt{X^TX}) = \sqrt{\rho}\\ \|D\| = \lambda_{max}(D) \leq 1 $$ So, all together, we have $$ \lambda_{max}(X^TDX) \leq \|X^TDX\| \leq \|X^T\|\cdot \|D\| \cdot \|X\| \leq \rho $$


Lower Bound:

First of all, note that $X^TDX$ is positive semidefinite, so its eigenvalues are certainly non-negative.

Let $d = d_{min}$. We note that the matrix $$ X^TDX - d(X^TX) = X^T(D - dI)X $$ is necessarily positive definite. It follows that $$ \lambda_{min}(X^TDX) \geq \lambda_{min}(dX^TX) = d\lambda_{min}(X^TX) $$ In fact, we could also have used this trick for the upper bound by noting that $X^T(\rho I - D)X$ is positive semidefinite.