deal with singular $A^TA$ in calculating pseuod inverse of A

Question

I want to calculate the pseudo-inverse of a rectangular matrix $A$ that is $A^{\dagger}=(A^TA)^{-1}A^T$, but I know that in my case $A^TA$ is a singular matrix and is not invertible. What's the solution to finding out the pseudo inverse of rectangular $A$ when $A^TA$ is singular?

Answer 1

Every matrix has a singular $A \in \mathbb{R}^{m \times n}$ has a singular value decomposition of the form $A = U \Sigma V^T$, where $U \in \mathbb{R}^{m \times m}$ and $V \in \mathbb{R}^{n \times n}$ are orthogonal matrices of left and right singular vectors respectively, and $\Sigma \in \mathbb{R}^{m \times n}$ is a diagonal matrix consisting of the singular values of $A$. A convenient representation of the Moore-Penrose pseudoinverse $A^{\dagger}$ is given by $A^{\dagger} = U \Sigma^{\dagger} V^T$, where $\Sigma^{\dagger} \in \mathbb{R}^{n \times m}$ is obtained by taking the transpose of $\Sigma$ and inverting the non-zero singular values.

Thus, in the special case that $A^T A$ is invertible ($A$ has linearly independent columns), $A^{\dagger}$ takes on the form $(A^T A)^{-1} A^T$ that you give.