Given two (in my case, real and symmetric) matrices $A$ and $B$ such that $A \succeq B$, I want to check if $\lVert A \rVert \geq \lVert B \rVert$ (for any matrix norm, but in my work I'm using the 2-norm). Here's my work so far:
Since $A - B \succeq 0$, we have, by definition, for some arbitrary $z \in \mathbb{R}^d$, $z^T (A - B) z \geq 0$. By Cauchy-Schwarz, we have
$$\begin{aligned} \langle z, (A-B)z\rangle &\leq \langle z, z\rangle \cdot \langle (A-B)z, (A-B)z\rangle \\ &= \lVert z \rVert^2_2 \cdot \lVert (A - B)z \rVert^2_2 \end{aligned}$$
But $z^T (A-B) z \geq 0$ so that $\lVert z \rVert^2_2 \cdot \lVert (A - B)z \rVert^2_2 \geq 0 \Rightarrow \lVert (A - B)z \rVert^2_2 \geq 0$ since $\lVert z \rVert^2_2$ is a non-negative real number. I'm not quite sure where to go from here. What am I missing?
ETA: In my problem, $B = \mu I$, but I'm wondering if there's a general relationship.
Yes, the spectral norm is just the largest singular value; since we are working over PSD matrices it is the same as the largest eigenvalue. But the largest eigenvalue of $A$ is larger than the largest eigenvalue of $B$: see here for a general answer.