Show that there exists a $v$ s.t. $A^TAv=\|A\|_2^2v$

Question

Given $A \in \mathbb{R}^{m \times n}$, show that there exists a $v$, s.t. $\|v\| = 1$ and $A^T A v = \|A\|_2^2 v$, where $$\|A\|_2 :=\max_{\|v\|=1}\|Av\|$$ is the spectral norm.

---

My attempt:

By the spectral theorem there exists $O \in O(\mathbb{R}^m): A^TA = O^TDO$, with $D$ diagonal. Now:

$$||A^TA||_2= \max_{||v||=1}||A^TAv||= \max_{||v||=1}\sqrt{v^TO^TDOO^TDOv}= \max_{||v||=1}||DOv||= \max_{||v||=1}||Dv||=|\lambda_{\max}|$$

With $\lambda_{\max}$ the largest eigenvalue by absolute value. So if I new that $\lambda_{\max}\geq 0$ I would know $\exists v: A^TAv=||A^TA||_2v$. Now I would only have to show that $||A||_2^2= ||A^TA||_2$, but somehow I can’t find a way to show that. I‘d be really thankful for some help:)

Answer 1

Because $A^TA$ is unitarily diagonalizable (i.e., the Spectral Theorem applies), it is easy to show that $$ \|A^TA\|=\sup\{\langle A^TAv,v\rangle:\ \|v\|=1\}=\sup\{\|Av\|^2:\ \|v\|=1\}=\|A\|^2. $$

For more details if $A^TA=U^TDU$ with $D$ diagonal, \begin{align} \sup\{\langle A^TAv,v\rangle:\ \|v\|=1\}&=\sup\{\langle D\,Uv,Uv\rangle:\ \|v\|=1\}\\[0.3cm] &=\sup\{\langle Dv,v\rangle:\ \|v\|=1\}\\[0.3cm] &=\|D\|=\|A^TA\|. \end{align}

Answer 2

The function $f(v)=\|Av\|$ is continuous on the compact sphere $S=\{v\colon \|v\|=1\}$ and therefore attains its maximum at some $v^* \in S$.

The function $$R(v)=\dfrac{\|Av\|}{\|v\|}$$ attains its maximum at $v^*$. The existence of $w^*$ such that $R(w^*)>R(v^*)$ is absurd because it would imply that $\|A\frac{w^*}{\|w^*\|}\|> \|Av^*\|$, contradicting the optimality of $v^*$.

Overall, we have $$\displaystyle\|A\| = \max_{v\in S}f(v)=\max_{v\neq 0}R(v)=R(v^*).$$

Now, as $v^*$ is a maximum of $R$, we have $\nabla R(v^*)=0$, which implies that $$\frac{A^TAv^*}{\|Av^*\|\|v^*\|}- \frac{\|Av^*\|v^*}{\|v^*\|^3}=0.$$ Rearranging the terms, we finally get $$A^TAv^*=\frac{\|Av^*\|^2}{\|v^*\|^2}v^*=R(v^*)^2v^*=\|A\|^2v^*.$$ The advantage of this approach is that it can be used to prove similar results for other norms (e.g. $\|A\|_{p,q}=\max_{\|w\|_p=1}\|Aw\|_q$ with $p,q\in (1,\infty)$)