Why is operator norm defined the way it is?

Question

Is there an intuition for the infimum definition of ${\| A \|}_{\mathrm{op}}$ without using a different, equivalent definition?

I am referring to the definition, given an operator $A: W \rightarrow V$, $$ {\| A \|}_{\mathrm{op}} = \inf \{ c \geq 0 : \| A v \| \leq c \| v \|, \ \forall v \in V \}.$$

---

Note that the $\sup$ definition, $$ \| A \|_{\mathrm{op}} = \sup \{ \| A v \| : v \in V, \| v \| =1 \},$$ agrees more with my intuition and I know that one can prove they are equivalent (as I have done here), but that exercise seems conceptually unrevealing.

---

To me the $\inf$ definition reads as choosing the smallest $c$ that bounds all vectors, however my current intuition thinks that its a better question to think about how large $A$ makes things, so rather a maximum.

I guess trivially this means things are always bounded by infinity but my point is that it doesn't really answer or correct my wrong intuition. I still feel the size of $A$ should be some sort of question about how it makes things large, so why do we consider the smallest?

Answer 1

The definition with the $\inf$ resembles the one used for constructing the Minkowski functional which, in turn, plays a key role in the proof of the Hahn-Banach theorem. The definition of the functional is carried out without using a norm, hence has meaning in any real vector space, normed or not. It is the use of the $\inf$ construct that allows to define the functional without the norm.

See, for example, Section 4.14 in Introductory Real Analysis, or the first chapter in Brezis's textbook on functional analysis, or Section 3.8 in Fundamentals of Functional Analysis.

Answer 2

In the first formulation, the $c$ gives an upper bound to how much the vector may grow (that's what $\le$ means). Taking the infimum then says that you want the lowest upper bound to the vector's growth.

Now remember what the supremum is: The supremum of a set is, by definition, the lowest upper bound to that set. In other words, by asking for the lowest upper bound, you are really asking for the supremum of the vector's growth.

Indeed, you can make this quite explicit in the formulas:

Start with the definition you gave:

$$\| A \|_{\rm op} = \inf \{ c \geq 0 : \| A v\| \leq c \|v\|, \forall v \in V \}$$

Now we observe that the zero vector doesn't constrain $c$ in any way. Therefore let's remove that from the considered set.

$$\| A \|_{\rm op} = \inf \{ c \geq 0 : \| A v\| \leq c \|v\|, \forall v \in V\setminus\{0\} \}$$

Now that we have excluded the zero vector, we see that $\|v\| > 0$, therefore we can divide the inequality by it without changing anything. This gives:

$$\| A \|_{\rm op} = \inf \left\{ c \geq 0 : \frac{\| A v\|}{\|v\|} \leq c , \forall v \in V\setminus\{0\} \right\}$$

But now the right hand side explicitly describes the lowest upper bound to $\|Av\|/\|v\|$, that is,

$$\| A \|_{\rm op} = \sup \left\{ \frac{\| A v\|}{\|v\|} : v \in V\setminus\{0\} \right\}$$

And here you've got your supremum.

Note again that in the last step, I did nothing but literally apply the definition of the supremum.

Answer 3

An informal, but hopefully intuitive, answer:

Write $\partial B_{r}$ for the set $\{ v \in V \ | \ \| v \| = r \}$. Think of the dotted hexagon as being $A(\partial B_{1})$ (the result of applying $A$ to $\partial B_{1}$), and think of the blue circle as being $\partial B_{{\| A \|}_{\mathrm{op}}}$.

$\hspace{6cm}$

To find ${\| A \|}_{\mathrm{op}}$, you can either find the biggest value of $\| A v \|$ for $v \in \partial B_{1}$, or the smallest $c$ such that $\partial B_{c} \supseteq A(\partial B_{1})$. The $\sup$ definition is the first method, and the $\inf$ definition is the second method.

The $\inf$ definition has $\partial B$ start out at infinity and shrink down to the smallest $c \geq 0$ so that $\partial B_{c}$ still contains $A(\partial B_{1})$, which is exactly the same as the smallest $c \geq 0$ so that $\| A v \| \leq c \| v \|$ $\forall v \in V$.