Prove that the discarded eigenvalues should be adjacent in Probabilistic PCA

30 Views Asked by At

I've been trying to understand Probabilistic PCA and I got stuck on the following equation that tries to find what should the discarded eigenvalues be

$$\mathcal{L} = -\frac{N}{2}\left\{d \ln(2\pi)+\sum_{j=1}^{q'}\ln\lambda_j+(d-q')ln\left(\frac{1}{d-q'}\sum_{j=q'+1}^{d}\lambda_j\right)+d \right\}$$

The authors argue that minimizing the above equation is same as minimizing

$$E := \ln\left(\frac{1}{d-q'}\sum_{j=q'+1}^{d}\lambda_j\right) - \frac{1}{d-q'}\sum_{j=q'+1}^{d}\ln\lambda_j$$

After this they say

Interestingly, the minimization of E leads only to the requirement that the discarded $\lambda_j$ be adjacent within the spectrum of the ordered eigenvalues of S

I'm unable to prove the above statement. I got some intuition from the following equivalent problem

$$E = \ln{x^Te} - \frac{1}{n}\sum_{i=1}^{n}\ln x_i$$

$$\nabla_x E = \frac{e}{x^Te} - \frac{1}{n}\sum_{i=1}^{n}\frac{e_i}{e_i^Tx}$$

A trivial solution occurs when all the components of $x$ are equal.

I'm unable to however convert this intuition into a rigorous solution, further is the trivial solution the only critical point?