Why do we use the supremum instead of maximum to define the norm of a matrix?

Question

Isn't $\{x:1=||x||\}$ always a closed set and aren't norms always continuous? If that's the case then why is the norm of a matrix defined as a supremum instead of as a maximum?

Answer 1

If $\max$ exists, then $\sup$ also exists and is the same value. So, one can replace $\max$ by $\sup$ in any context and never use $\max$. More consistent notation is generally a good thing. The flowchart for "when to use $\sup$ and when to use $\max$" simplifies to "use $\sup$". (*)

Similar to matrices, you will also often see $\|f\|_{C[a,b]} = \sup\{|f(x)|:x\in [a,b]\}$. This could be $\max$, but (a) does not really matter; (b) writing $\sup$ is consistent with what we would do for discontinuous functions and for functions on an unbounded interval, if such a space was considered there.

For matrices, using $\sup$ is consistent with what is done for operators on infinite-dimensional spaces.

(*) In practice, it's typical to use $\max$ for finite sets, where it's easier to think about picking the maximal element out of a finite collection. Also, $\max$ remains common when it really matters that the maximum is attained — for example, when we care about where it is attained.