I have a question to this post: Show adding rows to a non-singular square matrix will keep or increase its minimum singular value
There Tony asked how to show that the minimum singular value increases or stays the same when adding a row to a non-singular matrix. I'm also interested in how to show this. I understand the answer of loup blanc until he says that the last step is so obvious.
How do you know that $x^T(A^*_1A_1+A^*_2A_2)x\geq x^TA^*_1A_1x$ implies that the $spectrum(A^*_1A_1)\leq spectrum(A^*A)$? As far as I know, you can have a look at the inequality while using an eigenvector of matrix $A_1$ for example. But that doesn't mean that that same vector is also an eigenvector for $A$. So how do you know which one of the singular values is then smaller?
I also tried to describe the eigenvector of the smallest singular value of $A_1$ through a linear combination of the ones eigenvectors of $A$, but that didn't get me anywhere either.
I'd be grateful for any help because I think it shouldn't be that hard to show that this implication hold, but I just can't seem to see how to at the moment.
An explanation of LB's answer: first of all, his $x^T$ should really be an $x^*$ in this context. Let $\lambda_{\text{min}}(M)$ denote the smallest eigenvalue of a matrix $M$ (LB calls this "min spectrum" instead). Rayleigh's theorem says that if $M$ is Hermitian, we have $$ \lambda_{\min} (M) = \min_{\|x\| = 1} x^*Mx. $$ With that, we can say that $$ \lambda_{\min}(A_1^*A_1) = \min_{\|x\| = 1} x^*(A_1^*A_1)x \leq \min_{\|x\| = 1} x^*(A_1^*A_1 + A_2^*A_2)x = \lambda_{\min}(A_1^*A_1 + A_2^*A_2). $$
Here's an explanation that I find much more intuitive: $$ \begin{align} \sigma_{\min} \pmatrix{A_1\\A_2} &= \min_{\|x\| = 1} \left\| \pmatrix{A_1\\A_2}x\right\| = \min_{\|x\| = 1} \left\| \pmatrix{A_1 x\\A_2 x}\right\| \\&\geq \min_{\|x\| = 1} \left\| \pmatrix{A_1 x\\0}\right\| = \min_{\|x\| = 1} \|A_1x\| = \sigma_\min (A_1). \end{align} $$