How do I compute the Singular Value Decomposition of the pseudo-inverse of a matrix?

Question

There is $A$ which is a matrix: $$\begin{bmatrix}2 & 4 \\ 1 & -4 \\ -2 & 2\end{bmatrix}.$$ While I have easily worked out the singular value decomposition of this matrix, but I am not sure how to go about trying to present the pseudo-inverse of $A$ (i.e. $A^+$) in SVD form. What I have found out is that: $$ A^+ = V \cdot \Sigma^{-1} \cdot U^\top $$ But trying this out has caused a problem since the matrix $\Sigma$ is not a square matrix so the inverse is not possible. So I am not quite sure if I am following the right route or not ...

Answer 1

Just work out the pseudo inverse of $S$. You can do this by working out the inverse of the square portion of $S$ and then padding the matrix with zeroes to obtain the appropriate size.

Answer 2

You first transpose $S$ then replace the non-zero entries with their reciprocals to get $S^{-1}$.

Alternatively, you can use the thin SVD where $S$ is diagonal, so $S^{-1}$ just reciprocates the non-zero entries of the diagonal of $S$.

Answer 3

This is easier to deal with using dyadic (bra-ket) notation.

Write the SVD of $A$ as $$A=UDV^\dagger = \sum_k d_k u_k v_k^\dagger,$$ where $u_k$ (the left singular vectors) are the columns of $U$, $v_k$ (the right singular vectors) the columns of $V$, and $d_k\equiv D_{kk}>0$ the non-zero singular values.

The pseudo-inverse is then simply $$A^+ = \sum_k \frac{1}{d_k} v_k u_k^\dagger.$$ You switch left and right singular vectors and take the reciprocal of the singular values.

You can then also observe directly how $$A^+ A = \sum_k v_k v_k^\dagger = VV^\dagger = \text{projector onto the support of } A, \\ AA^+ = \sum_k u_k u_k^\dagger = UU^\dagger = \text{projector onto the range of }A.$$