Suppose $W=(X'X + kI)^{-1}$ and $Z=(I + k(X'X)^{-1})^{-1}$, $k>0$, and suppose also that $\lambda_i$ are eigenvalues of $X'X$. How to get the following conclusions about their eigenvalues.
- The eigenvalues of $W$ is $1 / (\lambda_i + k)$;
- The eigenvalues of $Z$ is $\lambda_i / (\lambda_i + k)$.
They are claimed from the original ridge regression paper.
The question might be trivial to you but I just had trouble working with inversion.
The basic point is that if $f$ is any rational function and $A$ is a matrix such that $f(A)$ is defined (i.e. none of the eigenvalues of $A$ is a pole of $f$), then $f$ maps the spectrum of $A$ to the spectrum of $f(A)$. Thus for (2.), we take $f(z) = 1/(1+k/z) = z/(z+k)$, and get the desired conclusion if $\lambda_i$ are the eigenvalues of $A = X'X$ and none of the $\lambda_i$ are $-k$ (which is the case if $k > 0$, $X$ is real, and $'$ denotes transpose).