Show that two definitions of matrix norm are in fact equivalent $\max\limits_{ x \neq 0} \dfrac{\|Ax\|}{\|x\|} = \max\limits_{\|x\| = 1} \|Ax\|$

Question

I found a proof in an online course note which purports to show that two definitions of matrix norms are equivalent, however, I have some doubts regarding the proof, I would like a second pair of eyes

Claim: $\|A\| = \max\limits_{ x \neq 0} \dfrac{\|Ax\|}{\|x\|} = \max\limits_{\|x\| = 1} \|Ax\|$

---

Proof:

$(\text{first show} \max\limits_{ x \neq 0} \dfrac{\|Ax\|}{\|x\|} \leq \max\limits_{\|x\| = 1} \|Ax\|)$

Suppose $x \neq 0$, then $\max\limits_{ x \neq 0} \dfrac{\|Ax\|}{\|x\|} = \max\limits_{ x \neq 0} \|A\dfrac{x}{\|x\|}\|$. Let $z = \dfrac{x}{\|x\|}$, $\|z\| = 1$. Therefore, $\max\limits_{ x \neq 0} \dfrac{\|Ax\|}{\|x\|} \leq \max\limits_{ \|z\| =1} \|Az\| = \max\limits_{ \|x\| =1} \|Ax\|$

-

$(\text{next show:}\max\limits_{ x \neq 0} \dfrac{\|Ax\|}{\|x\|} \geq \max\limits_{\|x\| = 1} \|Ax\|)$

Suppose that $\|x\| = 1$, therefore $\max\limits_{\|x\| = 1} \|Ax\| = \max\limits_{\|x\| = 1} \dfrac{\|Ax\|}{\|x\|} \leq \max\limits_{x \neq 0} \dfrac{\|Ax\|}{\|x\|}$, where the inequality follows from maximization over a superset of $\{x\in \mathbb{R}^n:\|x\|=1\}$.

-

Thus, $\max\limits_{ x \neq 0} \dfrac{\|Ax\|}{\|x\|} = \max\limits_{\|x\| = 1} \|Ax\|$.

It seems that there is no justification for $\max\limits_{ x \neq 0} \dfrac{\|Ax\|}{\|x\|} \leq \max\limits_{ \|z\| =1} \|Az\|$. It doesn't make sense, because we are maximizing over a much larger set than the unit circle. Can someone see if there is indeed a problem with the proof and if there is some opportunity to fix it?

Answer 1

Note that, if $x\neq0$, then$$\frac{\|Ax\|}{\|x\|}=\left\|A\left(\frac x{\|x\|}\right)\right\|$$and that $\left\|\frac x{\|x\|}\right\|=1$.

Answer 2

For $x\ne 0$, then $\dfrac{\|Ax\|}{\|x\|}=\|x\|^{-1}\|Ax\|=\|\|x\|^{-1}Ax\|=\left\|A\left(\dfrac{x}{\|x\|}\right)\right\|\leq\max_{\|z\|=1}\|Az\|$.