When I check for the proof of singular value decomposition, they all assume the following is true:
The image of the unit sphere under any $m * n$ matrix is a hyper ellipse.
However I could not find a decent proof for this, even though I googled for hours. I keep seeing notes like: "This geometric fact is not obvious. We shall restate it in the language of linear algebra and prove it later. For the moment, assume it is true."
Maybe I am using wrong keywords. Could you please give me a link, text book name, etc. (a reference) for this proof?
Suppose $T$ is a linear map on a finite-dimensional inner product space $V$. The Polar Decomposition states that there is an isometry $S$ on $V$ such that $$ T = S \sqrt{T^* T}. $$ Because $\sqrt{T^*T}$ is a positive operator, the Finite-Dimensional Spectral Theorem states that there is an orthonormal basis $e_1, \dots, e_n$ of $V$ and nonnegative numbers $s_1, \dots, s_n$ such that $$ \sqrt{T^*T}e_j = s_j e_j $$ for $j = 1, \dots n$. Thus $\sqrt{T^*T}$ maps the unit sphere of $V$ to a hyper-ellipse, and because $S$ is an isometry, $T$ also maps the unit sphere of $V$ to a hyper-ellipse.
The Singular Value Decomposition follows easily from the Polar Decomposition without mentioning hyper-ellipses (see, for example, Chapter 7 in my book Linear Algebra Done Right).