Comparing the norms of two matrix vector products.

92 Views Asked by At

Let $\| \cdot \|$ denote the operator norm of a matrix. Suppose $\|A\|\geq \|B\|$, can we conclude $\|Av\|_2\geq\|Bv\|_2$? Obviously the lower bound and upper bound you get for them (in terms of singular value) respect some inequalties but I am struggling to find results that let me rank these matrix vector products.

My intuition tells me no. For reason similar to this answer, as $B$ could potentially stretch a particular vector more than another. If this is indeed no, under what conditions on $A$ and $B$ can we rank $\|Av\|_2$ and $\|Bv\|_2$? In particular, the following might be useful $$ \|Av\|_2^2-\|Bv\|_2^2=v'(A'A-B'B)v>0 $$ The question then translates to under what conditions would $(A'A-B'B)$ be positive(semi) definite.

1

There are 1 best solutions below

1
On

You are correct that it is not generally true that $|Av|_2\geq|Bv|_2$ when $|A|\geq |B|$. A simple counterexample is $A=\begin{bmatrix}1&0\0&2\end{bmatrix}$, $B=\begin{bmatrix}0&1\1&0\end{bmatrix}$, and $v=\begin{bmatrix}1\0\end{bmatrix}$. Here, $|A|=2$, $|B|=1$, but $|Av|_2=1$ and $|Bv|_2=1$.

For the condition $v'(A'A-B'B)v>0$, it is not generally true that $(A'A-B'B)$ must be positive (semi) definite. A simple counterexample is $A=\begin{bmatrix}1&0\0&2\end{bmatrix}$, $B=\begin{bmatrix}1&0\0&1\end{bmatrix}$, and $v=\begin{bmatrix}1\0\end{bmatrix}$. Here, $v'(A'A-B'B)v=1$, but $(A'A-B'B)$ is not positive (semi) definite.

However, if $(A'A-B'B)$ is positive (semi) definite, then we can conclude that $|Av|_2\geq|Bv|_2$ for all vectors $v$. This follows from the Rayleigh-Ritz theorem, which states that for any matrix $M$ and any vector $v$, we have $|Mv|_2^2\geq v'M'Mv$. Therefore, if $(A'A-B'B)$ is positive (semi) definite, then $$ v'B'Bv\leq v'(A'A-B'B)v \leq v'A'Av $$ for all vectors $v$. This implies that $|Bv|_2^2\leq|Av|_2^2$, so $|Bv|_2\leq|Av|_2$.

Note that the condition $(A'A-B'B)$ being positive (semi) definite is not necessary for $|Av|_2\geq|Bv|_2$ to hold. It is possible for $|Av|_2\geq|Bv|_2$ to hold even if $(A'A-B'B)$ is not positive (semi) definite, as the countere