For what vectors does $A^+ A v = v$, where $A^+$ is the Moore-Penrose pseudoinverseof $A$?

Question

Let $A$ be an $m \times n$ matrix of rank $r$, and let $A^+$ be its Moore-Penrose pseudoinverse. For what vectors $\vec{v}$ (of length $n$) is it true that $A^+ A \vec{v} = \vec{v}$? What about in the special case where $A$ is square?

Answer 1

I think I figured it out.

Let the SVD of $A$ be $A = U \Sigma V^T$ (working with real matrices), where the elements of the diagonal matrix $\Sigma$ are sorted from largest to smallest. Then $A^+ = V \Sigma^+ U^T$, where $\Sigma^+$ is a diagonal matrix whose are the reciprocals of the corresponding elements of $\Sigma$ where they are non-zero and zero where the element of $\Sigma$ is zero.

Then $A^+ A = V \Sigma^+ U^T U \Sigma V^T$. Since $U$ is orthogonal by construction in SVD we may remove $U^T U$, leaving $A^+ A = V \Sigma^+ \Sigma V^T$. Let $\Lambda = \Sigma^+ \Sigma$. Then $\Lambda$ contains $r$ (the rank of $A$) ones along the diagonal, followed by zeroes. We are left with $A^+ A = V \Lambda V^T$, an eigendecomposition for $A^+ A$. The first $r$ vectors in $V$ will have eigenvalues of 1, forming a basis for the set of vectors $\vec{v}$ that satisfy $A^+ A \vec{v} = \vec{v}$. An interesting corollary is that $A^+A$ is (in the real case) symmetric.

Now assume $\vec{v}$ as in the row space of $A$, so that $\vec{v} = A^T \vec{x}$ for some $\vec{x}$. We can calculate

$$ A^+ A \vec{v} = V \Lambda V^T \vec{v} = V \Lambda V^T A^T \vec{x} = V \Lambda V^T (V \Sigma U^T) \vec{x}$$

And noting that $V^T V = I$ and $\Lambda \Sigma = \Sigma$ the expression becomes $V \Sigma U^T \vec{x} = A^T \vec{x} = \vec{v}$.

So we see that anything $\vec{v} = A^+ A \vec{v}$ if $\vec{v}$ is in the row space of $A$. And since that row space has dimension $r$, equal to the basis we found earlier for the set of vectors preserved by $A^+ A$, it is exactly the row space that is preserved.

Answer 2

This is a short note on Generalized inverse of any rectangular matrix $A \in \mathbb{C}^{m \times n}.$

$A^ \dagger : \mathbb{C}^{m} \to \mathbb{C}^{n}$ a linear transformation by $A^ \dagger $ defined as $A^ \dagger x=0 $ if $x \in R(A)^\perp$ and $A^ \dagger x=(A\mid_{R(A^*)})^{-1} x $ if $x \in R(A).$ Then the matrix $A^ \dagger$ denote the generalized inverse or Moore-Penrose inverse of $A.$ where $R(A), N(A), A^*$ denote the Range space , Null space and transpose conjugate of $A$.

Observations: $AA^\dagger x=0$ if $x \in R(A)^\perp$ and $AA^\dagger x=x$ if $x \in R(A).$

Similarly, $A^\dagger A x=0$ if $x \in N(A)=R(A^*)^\perp$ and $A^\dagger A x=x $ if $x \in R(A^*)=R(A^\dagger).$

(2) $AA^\dagger$ is the orthogonal projection of $\mathbb{C}^{m}$ onto $R(A)$ while $A^\dagger A$ is the orthogonal projection of $\mathbb{C}^{n}$ onto $R(A^*)=R(A^\dagger).$