Unable to calculate pseudo-inverse $A^TA$

Question

I'm trying to calculate the pseudo-inverse of $A^TA$ as described in this paper:

The SVD is particularly simple to calculate when the matrix is of the form $A^TA$ because $U=V$ and the rows of $U$ are the eigenvectors of $A^TA$ and the singular values in $D$ are the eigenvalues of $A^TA$. Since $U$ and $V$ are orthogonal matrices, the inverse of $M$ is then $M^{-1}=V^TD^{-1}U$

Since the pseudo-inverse of a non-singular matrix is it's inverse I tried to do this with a non-singular matrix and see if I get the inverse. I tried it with the following:

$A^TA = \begin{pmatrix} 90 & 51 \\ 51 & 29 \end{pmatrix}$ where $A = \begin{pmatrix} 3 & 2 \\ 9 & 5 \end{pmatrix}$

Eigenvalues and eigenvectors:

$\lambda_1 = 118.9243;$ $\lambda_2=0.0757;$ $v_1 = \begin{pmatrix} 1 \\ 0.5671 \end{pmatrix};$ $v_2 = \begin{pmatrix} 1 \\ -1.7632 \end{pmatrix}$

$U = V=\begin{pmatrix} 1 & 0.5671 \\ 1 & -1.7632 \end{pmatrix}$ $D = \begin{pmatrix} 118.9243 & 0\\ 0 & 0.0757 \end{pmatrix}$

And my result is:

$V^TD^{-1}U= \begin{pmatrix} 13.22 & -23.29 \\ -23.29 & 41.07 \end{pmatrix} \neq (A^TA)^{-1}$

Can someone help me? What am I doing wrong?

Answer 1

$V$ should be an orthogonal matrix and its two rows should be a set of two orthonormal eigenvectors of $A^TA$. Therefore the correct $V$ should be $\pmatrix{\frac{v_1^T}{\|v_1\|}\\ \frac{v_2^T}{\|v_2\|}}$, but you wrongly took $V$ as $\pmatrix{v_1^T\\ v_2^T}$ without normalising the eigenvectors.

Answer 2

I'm revising my previous post due to further research and my misconception on the pseudoinverse.

The pseudoinverse for a square matrix can exist even if the matrix is singular. If a square matrix is not singular the pseudoinverse is unique and should equal the inverse. I tested your example in Matlab and indeed the pseudoinverse and inverse are equal but not equal to your result.

The reason being is it appears your U and V matrices in the SVD are not correct. It should be

$U=V=\begin{bmatrix}-0.8698 & -0.4933\\-0.4933 & 0.8698\end{bmatrix}$