Why is $(X^{T}X+cI)^{-1} = (X^TX)^{-1}$ for ridge regression?

127 Views Asked by At

Why is $(X^{T}X+cI)^{-1} = (X^TX)^{-1}$? ($c$ is a scalar.)

My teacher in the lecture today used this fact when showing that the projection matrix of the ridge regression is idempotent. I was wondering if anyone knew of a proof they could share or an explanation. They wrote : $$X(X^T X + \lambda I)^{-1} X^T X (X^TX +\lambda I)^{-1}X^T = X(X^T X + \lambda I)^{-1} X^T$$ I was wondering why $(X^T X + \lambda I)^{-1} = (X^T X)^{-1}$ is true.

Edit: Had a miss understanding of what mathematical facts she was using, so the question has changed dramatically. Sorry guys I'm trying my best.

2

There are 2 best solutions below

5
On

The inverse of the inverse of an inversible matrix is the matrix itself so this equality can't be true unless $A + cI = A$ i.e. $c = 0$. Are you sure he wrote exactly that ?

4
On

The formula is wrong:
Just consider $X=I$ and $c=1$
You have
$$(X^TX+cI)^{-1}=(X^TX)^{-1}\\ (I^TI+1\cdot I)^{-1}=(I^T I)^{-1}\\ (I^2+ I)^{-1}=(I^2)^{-1}\\ (2I)^{-1}=I^{-1}\\ \frac{1}{2}I=I $$ But this is false.
also the formula $(X^TX+\lambda I)^{-1}=(X^TX)^{-1}$ is wrong (you can always use the same example)