As i know, both the left/right inverse matrix and singular value inverse matrix can get an inverse matrix of non square matrix.
Book shows that:
A matrix $A \in \mathbb{C}^{M \times N}$
if $M\geq N$, the left inverse matrix is $(A^{*}A)^{-1}A^{*}$;
if $M\leq N$, the left inverse matrix is $A^{*}(AA^{T})^{-1}$;
In the same time, $A$ can be decomposed as $UDV^{*}$, where $U$ and $V$ are unitary matrixs and $D$ is a diagonal matrix. $A^{-1}$ can be formulated as $VD^{+}U^{*}$.
Question: What is the difference among $(A^{*}A)^{-1}A^{*}$, $A^{*}(AA^{*})^{-1}$ and $VD^{+}U^{*}$.
Thanks a lot!
Left/right inverses only exist when the matrix $A_{m\times n}$ is full rank (i.e. $rank(A) = m$ when $A$ is a wide matrix and $rank(A) = n$ when $A$ is a tall matrix).
$$A^\dagger = \begin{cases}(A^TA)^{-1}A^T \qquad \text{rank(A) = n}\\A^T(AA^T)^{-1} \qquad \text{rank(A) = m} \end{cases}$$
The pseudoinverse $A^\dagger = VD^\dagger U^T$ derived from the SVD decomposition is the generalized inverse. You will notice that if you expand the $(A^TA)^{-1}A^T$ and $A^T(AA^T)^{-1}$ they will lead you right back to $A^\dagger = VD^\dagger U^*$. The main consideration is the structure of the $D$. For a full rank matrix A, $rank(A)=r=\min(m,n)$
$$D = \begin{cases} \left( \begin{array}{c} \begin{array}{ccc}\sigma_1 & &\\ & \ddots& \\ &&\sigma_r \end{array} \\ \hline \\0 \end{array}\right) = \left( \begin{array}{c} \Sigma \\0 \end{array}\right) & \qquad \text{if rank(A)=n} \\ \left( \begin{array}{c|c} \begin{array}{ccc}\sigma_1 & &\\ & \ddots& \\ &&\sigma_r \end{array} &0 \end{array}\right) = \left( \begin{array}{cc} \Sigma &0 \end{array}\right) & \qquad \text{if rank(A)=m} \end{cases}$$
$(A^TA)^{-1}$ and $(AA^T)^{-1}$ exist:
$$\begin{align}(A^TA)^{-1}&=\left(V\left(\begin{array}{cc}\Sigma&0\end{array} \right) \left(\begin{array}{c} \Sigma \\ 0\end{array}\right)V^T\right)^{-1} \\ &= V(\Sigma ^2)^{-1}V^T\end{align}$$
$$\begin{align}(AA^T)^{-1}&=\left(U\left(\begin{array}{cc}\Sigma&0\end{array} \right) \left(\begin{array}{c} \Sigma \\ 0\end{array}\right)U^T\right)^{-1} \\ &= U(\Sigma ^2)^{-1}U^T\end{align}$$
$$D^\dagger = \left( \begin{array}{c} \begin{array}{ccc}\sigma_1^{-1} & &\\ & \ddots& \\ &&\sigma_r^{-1} \end{array} \\ \hline \\0 \end{array}\right) = \left( \begin{array}{c} \Sigma^{-1} \\0 \end{array}\right)$$
However, for a rank deficient matrix (i.e. $rank(A)=r<min(m,n)$), the D becomes
$$D = \left(\begin{array}{c|c}\begin{array}{ccc} \sigma_1 &&\\&\ddots& \\&&\sigma_r \end{array}& \mathbf{0} \\ \hline \\ \mathbf{0} & \mathbf{0} \end{array}\right) = \left(\begin{array}{cc} \Sigma & 0 \\ 0&0\end{array} \right)$$
and $(A^TA)^{-1}$ and $(AA^T)^{-1}$ do not exist:
$$\begin{align}(A^TA)^{-1}&=\left(V\left(\begin{array}{cc}\Sigma&0\\0&0\end{array} \right) \left(\begin{array}{cc} \Sigma & 0 \\ 0&0\end{array}\right)V^T\right)^{-1} \\ &= V\left( \begin{array}{cc}\Sigma^2 & 0\\0&0 \end{array}\right)^{-1}V^T\end{align}$$
$$\begin{align}(AA^T)^{-1}&=\left(U\left(\begin{array}{cc}\Sigma&0\\0&0\end{array} \right) \left(\begin{array}{cc} \Sigma & 0 \\ 0&0\end{array}\right)U^T\right)^{-1} \\ &= U\left( \begin{array}{cc}\Sigma^2 & 0\\0&0 \end{array}\right)^{-1}U^T\end{align}$$
since $\begin{pmatrix} \Sigma^2 & 0 \\0&0 \end{pmatrix}^{-1}$ doesn't exist. That means for a rank-deficient matrix, the formula $(A^TA)^{-1}A$ and $A^T(AA^T)^{-1}$ cannot be used and we will have to resort to the generalized pseudoinverse which aims to find the best approximate inverse:
$$\begin{align}A^\dagger&= \mathop{\arg\min}_{\mathbf{X\in\mathbb{R}^{m\times n}}}||\mathbf{AX}-\mathbf{I}_m||_F \\&= \mathop{\arg\min}_{\mathbf{X}\in\mathbb{R}^{n\times m}}||\mathbf{XA}-\mathbf{I}_n||_F \\ &= V D^\dagger U^T\end{align}$$
where $$D^\dagger = \left(\begin{array}{c|c}\begin{array}{ccc} \sigma_1^{-1} &&\\&\ddots& \\&&\sigma_r^{-1} \end{array}& \mathbf{0} \\ \hline \\ \mathbf{0} & \mathbf{0} \end{array}\right) = \left(\begin{array}{cc} \Sigma^{-1} & 0 \\ 0&0\end{array} \right)$$