I am curious what is the precise math reasoning behind this:
$$ \sup \{ a_i^T u | \lVert u\rVert_2 \leq r \} = r \lVert a_i \rVert_2$$
It is on page 148, last line, of Boyd's Convex Optimization text.
I realize the statement is saying: the supremum of a dot b is equal to the length of a times the maximum length of b, but I was wondering if there was any other way to explain the reasoning.
Thank you all.
You ought to specify what space $a_i$ and $u$ are coming from, and what $\|\cdot\|_2$ is, but I'll assume the space is $\mathbb{C}^n$ or $\mathbb{R}^n$ and $\|\cdot\|_2$ is the Euclidean norm.
$$\sup \{a_i^Tu\ |\ \|u\|_2\leq r\} \geq a_i^T[(r/\|a_i\|_2)a_i] = r\|a_i\|_2$$
The key there was to realize the _dot product is maximized when $u$ is in the same direction as $a_i$_, so we take $u$ in the same direction as $a_i$ with length $r$. On the other hand:
$$\sup \{a_i^Tu\ |\ \|u\|_2\leq r\} \leq \{|a_i^Tu|\ |\ \|u\|_2\leq r\} \leq \{\|a_i\|_2\|u\|_2\ |\ \|u\|_2\leq r\} = r\|a_i\|_2$$
Where the second inequality follows from Cauchy-Schwarz.