Suppose we have $A$ with the size of $n \times d$, giving the SVD $A=V\Sigma U^T$, if $D$ is an identity matrix, ie $D=I$, then $(A^T A + \gamma I)^{-1}=U (\Sigma^2 + \gamma I)^{-1}U^T $.
If $D$ is not an identity matrix, but $D$ is a diagonal matrix with different positive entries, how can we efficiently compute $(A^T A + \gamma D)^{-1}$ if the dimension $d$ is very high??
All your help is much appreciated~
Consider \begin{align} (A^TA+\gamma D)^{-1}&=(A^TA+\gamma \sqrt{D}^T\sqrt{D})^{-1}\\ &=\sqrt{D}^{-1}(\sqrt{D}^{-T}A^TA\sqrt{D}^{-1}+\gamma I)^{-1}\sqrt{D}^{-T}\\ &=\sqrt{D}^{-1}[(A\sqrt{D}^{-1})^T(A\sqrt{D}^{-1})+\gamma I]^{-1}\sqrt{D}^{-T} \end{align} where $-T$ indicates matrix transpose after inverse or matrix inverse after transpose. Then the two problems will have the same form and thus they can be treated similarly.