How to show this matrix limit?

Question

Let $A$ be a $m\times n(m<n)$ matrix, $\mu>0$. $A$ is a full rank matrix. I am interested in showing that $$\lim_{\mu\to 0} (A^TA+\mu I)^{-1}A^T= A^T(AA^T)^{-1},$$ where $I$ is the identity matrix. Could you provide insights or a step-by-step proof for this limit expression?

Answer 1

The problem is equivalent to $$\lim_{\mu\to 0^+} (A^TA+\mu I)^{-1}A^TAA^T= A^T$$ i.e. (since $A$ is full rank) $$\lim_{\mu\to 0^+} (A^TA+\mu I)^{-1}A^TAA^TA= A^TA.$$ Since $A^TA$ is diagonalizable, this is equivalent to $$\lim_{\mu\to 0^+}\frac{\lambda^2}{\lambda+\mu}=\lambda$$ for each of its eigenvalues $\lambda$.

This last identity is obvious.

Answer 2

We can use singular value decomposition to show the limit. Assume $A=U[\Sigma_m, 0] V^T$, where $ U$ is a $ m\times m$ real orthogonal matrix, $\Sigma_m=\text{diag}(\sigma_1,\cdots,\sigma_m)$ is a $m\times m$ diagonal matrix with non-negative real numbers on the diagonal, V is a $n\times n$ real orthogonal matrix. So, \begin{align} &\lim_{\mu\to 0} (A^TA+\mu I)^{-1}A^T\\ =&\lim_{\mu\to 0} \left(V\left(\begin{bmatrix}\Sigma_m^T\Sigma &0 \\ 0&0\end{bmatrix}+I\right)V^T\right)^{-1}V\begin{bmatrix}\Sigma_m^T\Sigma &0 \\ 0&0\end{bmatrix}U^T\\ =&\lim_{\mu\to 0} \left(V\left(\begin{bmatrix}\sigma_1^2+\mu & \\ &\sigma_2^2+\mu \\ &&\ddots \\ &&& \sigma_m^2+\mu\\ &&&& \mu\\ &&&&& \mu\\ &&&&&&\ddots \\ &&&&&&&\mu \\ \end{bmatrix}+I\right)V^T\right)^{-1} V\begin{bmatrix}\Sigma_m^T\Sigma &0 \\ 0&0\end{bmatrix}U^T \\ =&\lim_{\mu\to 0} V\begin{bmatrix}\frac{\sigma_1}{\sigma_1^2+\mu} & \\ &\frac{\sigma_2}{\sigma_2^2+\mu} \\ &&\ddots \\ &&& \frac{\sigma_m}{\sigma_m^2+\mu} \\ &&&&0 \\ \end{bmatrix}U^T\\ =&V\begin{bmatrix}\Sigma_m^{-1}\\ &0 \\ \end{bmatrix}U^T\end{align}

and \begin{align} &A^T(AA^T)^{-1}\\ =&V\begin{bmatrix}\Sigma_m^T\Sigma &0 \\ 0&0\end{bmatrix}U^T (U\Sigma_m\Sigma^T_mU^T)^{-1}\\ =&V\begin{bmatrix}\Sigma_m^T\Sigma &0 \\ 0&0\end{bmatrix}U^T U(\Sigma_m\Sigma^T)^{-1}_mU^T\\ =&V\begin{bmatrix}\Sigma_m^{-1}\\ &0 \\ \end{bmatrix}U^T \end{align} So, $$\lim_{\mu\to 0} (A^TA+\mu I)^{-1}A^T =A^T(AA^T)^{-1}.$$

Answer 3

Let $\:A,B^T\in{\mathbb R}^{m\times n}\:$ and assume the existence of a function $f(z)$ which is well defined for both of the matrix arguments $(AB)$ and $(BA)$.

Then there is a wonderful (yet almost trivial) identity due to Higham $$f(BA)\;B = B\;f(AB)$$

In the current problem, take $f(z) = (z+\mu)^{-1}$ and $B=A^T$ to obtain $$\eqalign{ \lim_{\mu\to0}\;(A^TA+\mu I)^{-1}A^T &= \lim_{\mu\to0}\,A^T(AA^T+\mu I)^{-1} &= A^T(AA^T)^{-1} \\ }$$