Comparison of norms of positive definite matrices

Question

If $A \leq B$, where $A$ and $B$ are symmetric matrices, can we say that $ \| A \| \leq \|B\|$? In other words, If $A-B$ is negative semidefinite, does $ \| A \| \leq \|B\|$ hold?

Answer 1

This is obviously false, as $-I\preceq0$ but $\|-I\|>\|0\|$ for every norm.

There are counterexamples even if you require $A$ to be positive semidefinite. E.g. consider $$ A=\pmatrix{2&1\\ 1&1}, \ B=\pmatrix{3\\ &3}, \ D=\pmatrix{1\\ &4}, \ \|M\|=\|D^{-1}MD\|_\infty\tag{1} $$ where $\|M\|_\infty=\max_{\|v\|_\infty=1}\|Mv\|_\infty=\max_i\sum_j|m_{ij}|$ is the induced maximum norm. Then the norm $\|\cdot\|$ in $(1)$ is a matrix norm induced by the vector norm $\|x\|=\|D^{-1}x\|_\infty$. Now $0\prec A\prec B$ but $\|A\|=6>3=\|B\|$.

However, the inequality in question is true if $A$ is positive semidefinite and the matrix norm in use is unitarily invariant. (The induced $2$-norm, Frobenius norm and all Schatten norms fall into this category.)

Unitarily invariant norms can be characterised as symmetric gauge functions of singular values. When $A\preceq B$, the Courant-Fischer minimax principle dictates that $\lambda_i(A)\le\lambda_i(B)$ when the eigenvalues of $A$ and $B$ are arranged in the same (ascending or descending) order. So, when $A$ is also positive semidefinite, we have $\sigma_i(A)\le\sigma_i(B)$ for each $i$. Hence $f\left(\sigma_1(A),\ldots,\sigma_n(A)\right)\le f\left(\sigma_1(B),\ldots,\sigma_n(B)\right)$ for every symmetric gauge function $f$. In turn, $\|A\|\le\|B\|$ when the norm is unitarily invariant.