proof that $\|A^+\|_2 \leq\|A_1^{-1}\|_2$

Question

Suppose the $m\times n$ matrix $A$ has the form $$ A =\left[\begin{matrix}A_1\\ A_2\end{matrix}\right]$$ where $A_1$ is a nonsingular matrix of dimension $n\times n$ and $A_2$ is an arbitrary matrix of dimension $(m-n)\times n$. Let $A^+ = (A^*A)^{-1} A^*$ and show that $$ \|A^+\|_2 \leq \|A_1^{-1}\|_2.$$

Answer 1

Denote $B=AA_1^{-1}=\left[\matrix{I\\A_2A_1^{-1}}\right]$. A proof strategy (among several others) could be

1. Show that $B^*B-I$ is positive-semidefinite. It would mean that the eigenvalues $\lambda_i(B^*B)\ge 1$ and, hence, the singular values $\sigma_i(B)\ge 1$.
2. Show (e.g. via SVD) that $\|B^{+}\|_2=\frac{1}{\sigma_\min(B)}\le 1$.
3. Show that $B^{+}=A_1A^{+}$.
4. Estimate $$ \|A^{+}\|_2=\|A_1^{-1}A_1A^{+}\|_2\le\|A_1^{-1}\|_2\|A_1A^{+}\|_2\le \|A_1^{-1}\|_2. $$