Operator Norm of an augmented matrix vs unaugmented matrix

Question

This problem comes from Linear Algebra and its Applications by David C Lay and Steven R Lay.

Show that $$||[A B]|| \geq ||[A]||$$

Where $[A B]$ is a a matrix $A$ augmented with another matrix $B$ So if A = $$A = \begin{bmatrix} a & b \\ c & d\\ \end{bmatrix}$$ and $$B = \begin{bmatrix} e & f \\ g & h\\ \end{bmatrix}$$

Then $$[A B] = \begin{bmatrix} a & b & e & f \\ c & d & g & h\\ \end{bmatrix}$$

My attempt:

We know that the operator norm of a matrix is the square root of the largest eigenvalue of the matrix. What's really throwing me off here is that what if by augmenting the matrix $B$, the eigenvalues of $[A B]$ are less than that of just $A$.

I think the approach I should be going for starts with an SVD approach(?) $$[AB] = U_{AB}\Sigma_{AB}V^T_{AB}$$ $$[A] = U_{A}\Sigma_{A}V^T_{A}$$

From there I need to prove that the largest singular value of $[AB]$ is greater than that of $[A]$. Is this correct? Can someone share some insight?

Answer 1

You want to use the actual definition of the operator norm, which is $$ \|[AB]\| = \sup\{\|[AB]v\| : v\in \mathbb{R}^4 \text{ and } \|v\|=1\}$$ $$ \|A\| = \sup\{\|Av\| : v\in\mathbb{R}^2\text{ and } \|v\|=1\}.$$ Given any vector $(x,y)\in \mathbb{R}^2$, you have the vector $(x,y,0,0)\in \mathbb{R}^4$ with $[AB](x,y,0,0)$ = $A(x,y)$. From this, the set that $\|[AB]\|$ is the sup of contains the set that $\|A\|$ is the sup of.

Answer 2

Although an answer is already provided and what I have in mind seems the same, I am still going to write one in order to break the problem down a bit more.

---

The norms of operators over non-trivial (ie. not $\{\vec{0}\}$) normed linear spaces $T:(V,\|\cdot\|_v)\rightarrow(W,\|\cdot\|_w)$ is defined as

$$ \|T\|:=\sup\{\|T(v)\|_w:\|v\|_v=1\} $$

And that applies to the spectrum norm over $\mathbb{R}^n\rightarrow\mathbb{R}^m$ in your question as well.

For finite-dimensional vector spaces, the supremum is attained, meaning there is actually a particular vector $v_0$ such that $\|T(v_0)\|_v=\sup\{\|T(v)\|_w:\|v\|_v=1\}$. In that case, supremum becomes just maximum.

Using that fact, let $v_A$ be the vector where $\|A\|_2$ is attained. Note that $\|Av_A\|_2=\|A\|_2$.

Subscript 2 means 'spectrum norm'. We are only taking about one type of norm. From here, I am going to omit the subscript when I am referring to operator norm and keep the subscript for vector norm. Hopefully that makes reading easier for you.

Also note that the following vector has norm 1.

$$ w=\begin{bmatrix} v_A\\0 \end{bmatrix} $$

Thus $w$ belongs to the set of $v$ over which we are scanning for $\max \|[AB]v\|_2$ and so $\|[AB]w\|_2$ is smaller than or equal to the maximum out of the set (of vectors with norm 1).

$$ \therefore\|[AB]\|\geq\|[AB]w\|_2 $$

Now expand the matrix-vector multiplications.

$$ [AB]w=\begin{bmatrix} Av_A\\0 \end{bmatrix}\\ \therefore \|[AB]w\|_2=\|Av_A\|_2\\ $$

Combine that with the earlier inequality, we get

$$ \|[AB]\|\geq\|A\| $$