It is stated in the book Matrix Analysis by R. Horn (p. 453),
Let $A \in M_{n}(\mathbb{C})$. Let $A = V\Lambda W^{*}$ where $\Lambda = diag\{\sigma_{1}, \ldots, \sigma_{n}\}$ contains singular values of $A$ and $W = [w_{1}, \ldots,w_{n}]$. Then $$ \sigma_{i}(A) = \min\{\|Ax\| | x\in span\{w_{1}, \ldots w_{i}\}, \; \|x\|=1 \}. $$
I would like to prove this theorem.
My try is:
Considering $A = V\Lambda W^{*}$ we have
\begin{align} \min_{\qquad\|x\| = 1, \\ x\in span\{w_{1}, \ldots w_{i}\}} \|Ax\| &= \min_{\qquad\|x\| = 1, \\ x\in span\{w_{1}, \ldots w_{i}\}} \|V\Lambda W^{*}x\| \\ &= \min_{\qquad\|x\| = 1, \\ x\in span\{w_{1}, \ldots w_{i}\}} \|\Lambda W^{*}x\|\\ &= \min_{\qquad\|y\| = 1, \\ x\in span\{w_{1}, \ldots w_{i}\}} \|\Lambda y\|\\ \end{align} where $y=Wx$.
Now, I am not sure how to continue in a rigorous manner.
- Can you give me a hint to continue if my approach is correct?
- The $\min$ here should not be $\max$?
The point is that if $x=\sum_i c_i w_i$ then $Ax=\sum_i c_i \sigma_i u_i$. By the orthogonality you have $\| Ax \|^2 = \sum_i |c_i|^2 \sigma_i^2$, and by the unit assumption about $x$ you have that $\sum_i |c_i|^2=1$. Thus $\| Ax \|^2$ is a convex combination of the $\sigma_i^2$. Can you finish now?