Operator norm $||A||=\sup_{x \neq 0} \frac{||A(x)||}{||x||}$

Question

My teacher defined the operator norm as $||A||=\sup_{x \neq 0} \frac{||A(x)||}{||x||}$. I'm not really sure if I understand this definition, so, can anyone explain it to me, for example, let $A$ be $\begin{pmatrix}1&3\\ 5&7\end{pmatrix}$. Or if it has to be with functions, let $A$ be $\begin{pmatrix}x&3x\\ 5x&7x\end{pmatrix}$.

Thank you.

Answer 1

Using the example provided, the operator norm of $A$ is the quantity, $$\sup_{(x, y) \neq (0, 0)} \frac{\left\|\begin{pmatrix} 1 & 3 \\ 5 & 7\end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix}\right\|}{\left\|\begin{pmatrix} x \\ y \end{pmatrix}\right\|} = \sup_{(x, y) \neq (0, 0)} \frac{\left\|\begin{pmatrix} x + 3y \\ 5x + 7y \end{pmatrix}\right\|}{\left\|\begin{pmatrix} x \\ y\end{pmatrix}\right\|} = \sup_{(x, y) \neq (0, 0)} \sqrt{\frac{(x + 3y)^2 + (5x + 7y)^2 }{x^2 + y^2}}.$$

Answer 2

Let's look at an example: take $$ A = \pmatrix{3&1\\1&3} $$ we have $$ A(x) = \pmatrix{3&1\\1&3}\pmatrix{x_1\\x_2} = \pmatrix{3x_1+x_2\\x_1+3x_2} $$ So, we have $\|Ax\| = \sqrt{(3x_1+x_2)^2 + (x_1+3x_2)^2}$. Our goal is to compute $$ \sup_{x\neq 0} \frac{\|Ax\|}{\|x\|} = \sup_{(x_1,x_2) \neq (0,0)}\frac{\sqrt{(3x_1+x_2)^2 + (x_1+3x_2)^2}}{\sqrt{x_1^2 + x_2^2}} $$ For this operator, we can make things easier to compute with the substitution $u_1 = x_1 + x_2$ and $u_2 = x_1 - x_2$. With this we can write $$ \frac{\sqrt{(3x_1+x_2)^2 + (x_1+3x_2)^2}}{\sqrt{x_1^2 + x_2^2}} = \frac{\sqrt{25u_1^2 + u_2^2}}{\sqrt{u_1^2+u_2^2}} $$ So that $$ \sup_{(x_1,x_2)\neq (0,0)} \frac{\sqrt{(3x_1+x_2)^2 + (x_1+3x_2)^2}}{\sqrt{x_1^2 + x_2^2}} = \sup_{(u_1,u_2) \neq (0,0)}\frac{\sqrt{25u_1^2 + u_2^2}}{\sqrt{u_1^2+u_2^2}} $$ so that in this case, $\|A\| = 5$.