Show that the Moore-Penrose pseudoinverse has the smallest frobenius norm

374 Views Asked by At

Let $A$ be an $N \times d$ matrix and $A^+ = (A^TA)^{-1}A^T$ be the Moore-Penrose Pseudoinverse of $A$. If $L$ is another $d \times N$ left inverse of $A$, then $L = A^+ + E$ for some $E$ such that $EA=0$. We define the Frobenius norm as $||A||_F=\sqrt{\overset{N}{\underset{i=1}{\sum}}\overset{d}{\underset{j=1}{\sum}}A_{i,j}^2}$. Show that $||A^+||_F \le ||L||_F$.

Here's what I have: We know that $||A||_F=\sqrt{tr(A^TA)}$. Using that $L=A^++E$, we can say that:

$||L||_F^2=tr((M+E)^T(M+E))$

$\ \ \ \ \ \ \ \ \ =tr((M^T+E^T)(M+E))$

$\ \ \ \ \ \ \ \ \ =tr(M^TM + E^TM + M^TE + E^TE)$

$\ \ \ \ \ \ \ \ \ =tr(M^TM) + tr(E^TM) + tr(M^TE) + tr(E^TE)$

This is where I'm stuck. I think I need to show that this expression is equivalent to

$tr(M^TM) + tr(E^TE)$

Since then it will just be $||M||_F^2$ plus a positive constant, but that requires showing that $tr(E^TM) + tr(M^TE)$ are $0$ which I'm not sure how to do.

1

There are 1 best solutions below

3
On

First, notice that $tr(M^TE)=tr((M^TE)^T)=tr(E^TM)$. So $tr(E^TM)+tr(M^TE)=2tr(M^TE)$.

Next, since $M=A^+=(A^TA)^{-1}A^T$, we have $M^T=A((A^TA)^{-1})^T$. So $$tr(M^TE)=tr(A((A^TA)^{-1})^TE)=tr(((A^TA)^{-1})^T\stackrel{=\ 0}{\overbrace{EA}})=0$$