If $A$ is a non-square matrix with orthonormal columns, what is $A^+$?

Question

If a matrix has orthonormal columns, they must be linearly independent, so $A^+ = (A^T A)^{−1} A^T$ . Also, the fact that its columns are orthonormal gives $A^T A = I$. Therefore,

$$A^+ = (A^T A)^{−1} A^T = (I)^{-1}A^T = A^T$$

Thus, $A^+ = A^T$. Am I correct? Thank you.

Answer 1

The pseudoinverse $\rm A^+$ must satisfy $4$ conditions:

• $\rm A A^+ A = A$

• $\rm A^+ A A^+ = A^+$

• $\rm (A A^+)^* = A A^+$

• $\rm (A^+ A)^* = A^+ A$

Let us verify that $\color{blue}{\rm A^+ = A^\top}$. The first two conditions are satisfied because

$$\rm A A^+ A = A \underbrace{\left( A^\top A \right)}_{=I} = A$$

and

$$\rm A^+ A A^+ = \underbrace{\left( A^\top A \right)}_{=I} A^\top = A^\top = A^+$$

and the last two conditions are trivially satisfied if $\rm A$ is a real matrix.

Answer 2

Problem statement

Start with a matrix $$A\in\mathbb{C}^{m\times n}$$ where $m>n$, and a valid statement for the pseudoinverse matrix $$ \mathbf{A}^{+} = \left( \mathbf{A}^{*} \mathbf{A} \right)^{-1} \mathbf{A}^{*} $$ We know (see links) that this matrix is a left inverse: $$ \mathbf{A}^{+} \mathbf{A} = \mathbf{I}_{n} \tag{1} $$

If we add the constrain that the column vectors of $\mathbf{A}$ are orthonormal, we also have $$ \mathbf{A}^{*} \mathbf{A} = \mathbf{I}_{n} \tag{2} $$

Conclusion

The results $(1)$ and $(2)$ suggest the identity $$ \mathbf{A}^{+} \mathbf{A} = \mathbf{I}_{n} = \mathbf{A}^{*} \mathbf{A} $$ from which we conclude that $$ \mathbf{A}^{+} = \mathbf{A}^{*} $$

Example

$$ \mathbf{A} = \frac{1}{\sqrt{2}} \left[ \begin{array}{cr} i & -1 \\ i & 1 \\ 0 & 0 \\ \end{array} \right], \qquad \mathbf{A}^{*} = \frac{1}{\sqrt{2}} \left[ \begin{array}{rrc} -i & -i & 0 \\ -1 & 1 & 0 \\ \end{array} \right] $$

---

$$ \mathbf{A}^{+} \mathbf{A} = \left[ \begin{array}{cc} 1 & 0 \\ 0 & 1 \\ \end{array} \right] = \mathbf{I}_{2}, \qquad \mathbf{A} \mathbf{A}^{+} = \left[ \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \\ \end{array} \right] \ne \mathbf{I}_{3} $$

---

$$ \mathbf{A}^{*} \mathbf{A} = \left[ \begin{array}{cc} 1 & 0 \\ 0 & 1 \\ \end{array} \right] = \mathbf{I}_{2}, \qquad \mathbf{A} \mathbf{A}^{*} = \left[ \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \\ \end{array} \right] \ne \mathbf{I}_{3} $$

---

Background reading

Categorize the pseudoinverse matrix in terms of left and right inverses: generalized inverse of a matrix and convergence for singular matrix, What forms does the Moore-Penrose inverse take under systems with full rank, full column rank, and full row rank?

General properties of the pseudoinverse matrix: Moore–Penrose pseudo-inverse Reference.

Answer 3

If the real matrix $A$ has a left inverse, then $A^+=(A^TA)^{-1}A^T$.

Note that $A^TA$ being invertible is equivalent to $A$ having a left inverse.

If the columns of $A$ are orthonormal, then $A^TA=I$ and therefore $A^+=A^T$.

The initial statement is just a verification of the required properties for the pseudoinverse.

---

Suppose on the contrary that $A^+=A^T$. Consider $A=QR$, where $Q$ has orthonormal columns and $R$ has a right inverse (such a decomposition can be easily derived from the Gram-Schmidt algorithm). Then, by general theory, $$ A^+=R^+Q^+=R^T(RR^T)^{-1}Q^T=A^T=R^TQ^T $$ If we multiply by $Q$, we obtain $R^T(RR^T)^{-1}=R^T$ and therefore $$ RR^T=I $$ Thus $A$ is the product of a matrix with orthonormal columns and a matrix with orthonormal rows (the transpose of a matrix with orthonormal columns, if you prefer).