Let $A$ be a $n \times n$ matrix ($A \in \mathbb{R}^{n \times n}$). Then for each $x \in \mathbb{R}^n$ the vector $Ax$ is defined and so we can see the matrix $A$ as a function $A: \mathbb{R}^n \to \mathbb{R}^n$.
We define $\mathcal{B}=\{ M: \|Ax\|_2 \leq M \|x\|_2 \text{ for each } x \in \mathbb{R}^n\}$.
If $A \neq 0$ then $\inf \mathcal{B} >0$ and we define :
$$\|A\|:= \inf \{ M>0: \|Ax\|_2 \leq M \|x\|_2 \forall x \in \mathbb{R}^n\}$$
I want to prove that the following holds:
$$\|A\|= \sup \left \{ \frac{\|Ax\|_2}{\|x\|_2}: x \in \mathbb{R}^n \setminus{\{0\}} \right\}\\ = \sup \{ \|Ax\|_2:\|x\|_2 \leq 1 \} \\ = \sup \{ \|Ax\|_2: \|x\|_2=1\}$$
I have tried the following.
We have that $$\|A\|= \inf \{ M>0: \|Ax\|_2 \leq M \|x\|_2 \forall x \in \mathbb{R}^n\}$$
For $\|x\|_2 \neq 0$ we get $$\|Ax\|_2 \leq M \|x\|_2 \Rightarrow M \geq \frac{\|Ax\|_2}{\|x\|_2}$$
But then how do we get to the supremum of $\frac{\|Ax\|_2}{\|x\|_2}$ ?
In order to prove that $\sup \{ \|Ax\|_2: \|x\|_2 \leq 1\}= \sup \{ \|Ax\|_2: \|x\|_2=1\}$ I have thought the following:
$$\{ \|Ax\|_2: \|x\|_2=1\} \subseteq \{ \|Ax\|_2: \|x\|_2 \leq 1\} \\ \Rightarrow \sup \{ \|Ax\|_2: \|x\|_2=1\} \leq \sup \{ \|Ax\|_2: \|x\|_2 \leq 1\} $$
So it remains to show that $\{ \|Ax\|_2: \|x\|_2 \leq 1\} \leq \sup \{ \|Ax\|_2: \|x\|_2=1\}$.
But how can we show this?
Also it holds that $\{ \|Ax\|_2: \|x\|_2=1\} \subseteq \left \{ \frac{\|Ax\|_2}{\|x\|_2}: x \in \mathbb{R}^n \setminus{\{0\}} \right\} $, right?
Thus $\sup \{ \|Ax\|_2: \|x\|_2=1\} \leq \sup \left \{ \frac{\|Ax\|_2}{\|x\|_2}: x \in \mathbb{R}^n \setminus{\{0\}} \right\}$.
But again how can we show that $\sup \left \{ \frac{\|Ax\|_2}{\|x\|_2}: x \in \mathbb{R}^n \setminus{\{0\}} \right\} \leq \sup \{ \|Ax\|_2: \|x\|_2=1\}$ ?
Let $\|A\|$ be defined via the infimum. Then for $x\ne0$, $\frac{\|Ax\|}{\|x\|}\le\|A\|$, so $\sup_{x\ne0}\frac{\|Ax\|}{\|x\|}\le\|A\|.$ On the other hand, if $N=\sup_{x\ne0}\frac{\|Ax\|}{\|x\|}$, then $\|Ax\|\le N\|x\|$ for all $x$, so $\|A\|\le N$ and equality holds.
As for the other equalities, if $x\ne0$ then $\|x\|>0$, so $$\sup_{x\ne0}\frac{\|Ax\|}{\|x\|}=\sup_{x\ne0}\left\|A\frac x{\|x\|}\right\|=\sup_{\|u\|=1}\|Au\|.$$
Now, let $x=cu$, where $0<c\le1$ and $\|u\|=1$. Then $\|Ax\|=\|cAu\|=c\|Au\|\le\|Au\|$, so $\sup_{\|x\|\le1}\|Ax\|=\sup_{\|x\|=1}\|Ax\|$.