I was wondering if someone could explain me the concept of the norm of a linear application or a matrix.

Question

I was wondering if someone could explain me the concept of norm. For example, if I have a linear application from $\mathbb{R}^2 \to \mathbb{R}$, I understand that the norm is the maximum slope of the graph (plane) of the application. But when it comes to applications in other dimensions, I cannot visualize it and I don't get the intuition on what is this norm.

Answer 1

As others have pointed out, you can't visualize things in above three dimensions. But there are plenty of other intuitions to be had besides visual ones.

Probably the best way to think about the norm of a linear operator is as a control on the magnitude of the image vectors, compared to the preimage vectors. To put it more simply, if you have a continuous linear operator $A:X\to Y$ where $X$ and $Y$ are normed spaces, then $\|Ax\|_Y\leq\|A\|_{\text{op}}\|x\|_X$ for all $x\in X$. So, the operator $A$ can't "blow up" any vectors beyond the scale of its operator norm.

In the finite-dimensional case, there are other matrix norms besides just the operator norm; but they are all equivalent so you can still think of them intuitively as above.

Answer 2

When you think of a matrix $A$, you should think of the linear transformation $x \mapsto Ax$. A natural way to measure the "size" of $A$ is to ask, how large can the output $Ax$ possibly be, given that the input $x$ has norm $1$. So, the operator norm of a real $m \times n$ matrix $A$ is defined to be $\sup \{ \| Ax \| \mid x \in \mathbb R^n, \| x \| = 1 \}$.