Operator norm $4$ different definitions how to prove that $\sup$ is $\max$ and $\inf$ is $\min$ for the last two?

Question

From what I have understood, all the $\sup$'s and $\inf$'s in the $4$ different definitions of the operator norm can be taken as $\max$'s and $\min$'s.

For a linear map $A\in \mathscr L (V,W)$ between two vector spaces $V$ and $W$ (of the form $\mathbb R^k$): $$\begin{align*} I &= \inf\{ c\;\colon\; \lVert Av\rVert\leq c\lVert v\rVert \text{ for all }v\in V\}\\ S_1&=\sup\{ \lVert Av\rVert\;\colon\; v\in V\text{ with }\lVert v\rVert\leq 1\}\\ S_2&=\sup\{\lVert Av\rVert\;\colon\; v\in V\text{ with }\lVert v\rVert = 1 \}\\ S_3&=\sup\left\{ \frac{\lVert Av\rVert}{\lVert v\rVert}\;\colon\; v\in V\text{ with }v\neq 0\right\}. \end{align*}$$

As $\{ x\in V \mid ||x|| \leq 1\}$ and $\{ x\in V \mid ||x|| = 1\}$ are compact, as the composition of the absolute value with $A$ is continuous and as this composition is a real-valued function, we can conclude that the $\sup$ is attained, and hence is the same as the $\max$ for $S_1$ and $S_2$.

For $S_3$ , I don't see why the $\sup$ would be equal to the $\max$. The function considered is $|\frac{A}{Id}|$ and the domain is $V\backslash \{0\}$. As far as I know, $V\backslash \{0\}$ is not compact (since it is unbounded).

I have the same problem for $I$ (proving that the $\inf$ is the $\min$).

Thanks in advance.

PS: I also am wondering if the vector spaces $V$ and $W$ can be of an other form than $\mathbb R^k$. But I doubt so, because then we could not argue that $\{ x\in V \mid ||x|| \leq 1\}$ and $\{ x\in V \mid ||x|| = 1\}$ are compact.

Answer 1

To address your first question: the $\sup$ can be turned into a $\max$ because the set $$\left\{\frac{\|Ax\|}{\|x\|} : x\neq 0\right\} = \{\|Ax\| : \|x\| = 1\}$$ is compact.

To address your second question, indeed we can consider other spaces than $\mathbb R^n$. One such type of space is called a Banach space. Formally, a Banach space is a vector space $V$ equipped with a norm $\|\cdot\|$ such that the metric $d(x,y) := \|x-y\|$ generates a complete metric space. Frequently encountered Banach spaces are the $L^p$ spaces on $\mathbb R^n$, i.e., $$L^p(\mathbb R^n) := \left\{f : \mathbb R^n \to \mathbb R : \int_{\mathbb R^n} |f|^p\,d\mu < \infty\right\},$$ with the associated norm $\|\cdot\|_p$ defined by $$\|f\|_p := \left(\int_{\mathbb R^n} |f|^p\,d\mu\right)^{1/p}.$$ Here $\mu$ denotes the Lebesgue measure. In the special case of $L^2$ it turns out that $\|\cdot\|_2$ is generated by the inner product $$(f,g) = \int_{\mathbb R^n} fg\,d\mu, \quad \|f\|_2 = \sqrt{(f,f)}.$$ This leads us to a second popular way to study spaces other than $\mathbb R^n$: Hilbert space. A Hilbert space $V$ is a Banach space where the norm $\|\cdot\|$ is generated by an inner product; i.e., there is an inner product $(\cdot,\cdot)$ such that $\|x\| = \sqrt{(x,x)}$ for all $x\in V$.

It is still possible (and in fact, useful) to define operator norms on a Banach space $V$. In fact, they are defined exactly the same. In particular, if $T : V \to V$ is a linear operator, then $\|T\|_{op} < \infty$ if and only if $T$ is continuous, where $\|\cdot\|_{op}$ is the operator norm.

Indeed, your observation that $\{x \in V : \|x\| = 1\}$ is not compact holds whenever $V$ is infinite dimensional, by Riesz's lemma.

Answer 2

I've finally found a way (my way) of proving that for $I$, the $\inf$ is equal to the $\min$:

$\{ c\;\colon\; \lVert Av\rVert\leq c\lVert v\rVert \text{ for all }v\in V\}$ $=$ $\{ c\;\colon\; \lVert Av\rVert\leq c \text{ for all }v\in V,\text{ such that } \lVert v \lVert = 1\}$ (to see this, just divide the inequality by $\lVert v \lVert$).

The last set is nothing more than $F^{-1} ([0,c])$ with $F = \lVert A \lVert$ (the composition of absolute value with $A$), intersected with the compact set $\{v\in V \text{ such that } \lVert v \lVert = 1\}$.

Hence $\{ c\;\colon\; \lVert Av\rVert\leq c\lVert v\rVert \text{ for all }v\in V\}$ is a part of $\mathbb R$, bounded from below (by $0$) and closed ; we conclude that $\inf\{ c\;\colon\; \lVert Av\rVert\leq c\lVert v\rVert \text{ for all }v\in V\}$ belongs to the set and hence is a $\min$.