Kernel PCA uses the kernel trick to perform PCA in a richer (RKHS) feature space. However, I'm confused about whether it's precisely the standard PCA algorithm applied in a larger space, or if there's a subtle difference in the eigenvalue problem.
Bishop, p588 derives the eigenfunction problem as $Cv_i = \lambda_i v_i$ for $C = \frac{1}{N}\sum_{n=1}^N \phi(x_n)\, \phi(x_n)^T$ (this is an abuse of notation if $\phi(x_n)$ is a function). Next, Bishop expresses $v_i = \sum_{n=1}^N a_{in} \phi(x_n)$ and substitutes $v_i$ into the EVP to derive $K^2 a_i = \lambda_i N K a_i$ for kernel matrix $K$. Please let me know if this is the wrong interpretation, but this somehow feels like a dual representation of $v_i$ and therefore some sort of "dual" EVP where instead of working in infinite dimensions, we're working in $N$ dimensions (the dimension of $a_i$). So:
- Is this the right interpretation?
- And if so, is this a meaningful difference?