Inverse of sum of matrices (SVD, ridge regression)

Question

Looking at these slides, I've found the following:

$X=UDV^T$, where $U$ and $V$ are orthogonal matrices, $V$ is a square matrix, and $D$ contains the singular values of $X$.

The author then writes

$$X(X^TX + \lambda I)^{−1}X^Ty$$ $$=UD(D^2 + \lambda I)^{−1}DU^Ty.$$

Why does that equality hold? I've tried substituting $X=UDV^T$, have got

$$UDV^T(VD^2V^T+\lambda I)^{-1}VDU^Ty.$$

If there wasn't the $+\lambda I$ term, then, as $V$ is a square matrix, we would have $$UDV^T V^{-T}D^{-1}D^{-1}V^{-1} VDU^Ty$$ $$=UU^Ty.$$

However, it's not the case that there isn't the $+\lambda I$ term, so I don't know how to proceed.

Answer 1

I'm answering this on a cell phone so pardon if I don't use lots of notation, but you're essentially just one step away from the answer with your substitution. Note that $$(VD^2V^T+\lambda I)^{-1}=(VD^2V^T+V\lambda IV^T)^{-1}=(V(D^2+\lambda I)V^T)^{-1}$$ From here the final step will look like cheating, but you can actually pull out the $V$ and $V^{T}$ because inversion reverses products. We get $$(V(D^2+\lambda I)V^T)^{-1}=V(D^2+\lambda I)^{-1}V^T$$ Plug this in to get what you want.

Answer 2

Here is the point :

In: $\ UDV^T(VD^2V^T+\lambda I)^{-1}VDU^Ty$

you should write $I=VV^T$ giving the factorization:

$UDV^T(V(D^2+\lambda I)V^T)^{-1}VDU^Ty$

$=UDV^T(V^T)^{-1}(D^2+\lambda I)^{-1}V^{-1}VDU^Ty$

yielding the looked-for formula.