Let $X$ be a $m \times n$ matrix. By SVD, I obtain $X = UDV^T$, where $U$ and $V$ are both orthogonal matrices, and $D$ is a diagonal matrix.
I think the following is true (but not sure why):
$(VDV^T + aI_n)^{-1} = V(D + aI_n)^{-1}V^T$
where $a\in \mathbb{R}$ is some scalar, and $I_n$ is a $n\times n$ identity matrix. Can anyone explain to me why one is able to move $V$ to the left of the matrix inverse and $V^T$ to the right?
I don't know what the SVD has to do with it but yes, your manipulation is correct.
We are given that $D$ is diagonal and $V$ is orthogonal. Note that $V^{-1} = V^T$ and $(V^T)^{-1} = V$. So, we have $$ \begin{align} (VDV^T + aI_n)^{-1} &= (VDV^T + V(a I_n)V^T)^{-1} \\ & = (V(D + aI_n)V^T)^{-1} \\ & = (V^T)^{-1}(D + aI_n)^{-1}V^{-1} = V(D + aI_n)^{-1}V^T. \end{align} $$