Compactness argument in SVD existence proof

Question

The classical proof of the existence of the SVD factorization by Trefethen and Bau reports

Set $\sigma_1 = \mid\mid A \mid\mid_2$. By a compactness argument, there must be a vector $v_1 \in \mathbb{C}^n$ with $\mid\mid v_1 \mid\mid = 1$ and $\mid\mid u_1 \mid\mid_2 = \sigma_1 $ where $u_1 = A v_1$.

where $A$ is a complex matrix of size $m \times n$.

Because it is presented in such a brisk fashion, I expect it to be something very elementary, but I cannot follow the reasoning at all. I guess that we are interested in the compactness of $\mathbb{C}^n$, but what are the implications of compactness which are relevant in this case?

Thanks!

Answer 1

It depends on how the induced matrix norm was defined. I don't have the book handy, but I expect some pages earlier the authors to have put $$\|A\|_2=\sup_{\|v\|_2=1}\|Av\|_2.$$ Note that a function's sup is not always achieved (think of $1-\exp x$ ($x$ real) and $1$). Compactness (for your purposes here) is a quick way of saying that the sup is achieved by a vector $v$, i.e., there is a specific vector $v_1$ which satisfies $$\|v_1\|_2=1\text{ and }\|Av_1\|=\|A\|_2.$$

Answer 2

We are interested in the compactness of the subset $S = \{v\mid \lVert v \rVert = 1\}\subseteq \Bbb C^n$. Compactness is relevant because among those vectors, $||Av||_2 \leq \sigma_1$, and compactness is used to guarantee the existence of a vector $v_1$ such that there is equality. In other words, the function $f:S \to \Bbb R$ given by $f(v) = \lVert Av\rVert_2$ has sup $\sigma_1$ by definition of operator norm, and compactness guarantees that it is actually a max.

Answer 3

I have question about the same proof. In the book it mentioned

"Set $σ_{1}=\|A\|_{2}$.By a compactness argument, there must be vectors $v_{1} \in C^n$ and $u_{1} \in C^{m}$ with $\|v_{1}\|_{2}=\|u_{1}\|_{2}=1$ and $Av_{1}=σ_{1}$." Since it was defined earlier that $\|A\|_{2}=sup_{\|v\|_{2}=1} \|Av\|_{2}$ and $σ_{1}\geqslantσ_{2}\geqslant…$ , as semi-axis principle, I can follow why $σ_{1}=\|A\|_{2}$ but how it came to the conclusion that there is $u_{1}\in C^{m}$ with $\|u_{1}\|_{2}=1$ and $Av_{1}=σ_{1}u_1$.