Why is the product of two norms is always bigger or equal to the norm of the same corresponding element?

Question

E.g. if $A$ is a matrix and $v$ is a vector which can be multiplied with the matrix, it always applies that (no matter how the norm is defined):

$||A|| ||v||\geq ||Av||$

Why is it so?

Answer 1

Your inequality contains three norms: $\|A\|$, $\|v\|$ and $\|Av\|$. The last two are the norm of a vector, respectively $v$ and $Av$. You are right that you can use any norm here. But once you decide for one such norm then $\|A\|$ is defined by the formula

$$\|A\| = \max_{w\neq 0}{\frac{\|Aw\|}{\|w\|}}$$

i.e. the norm on matrices depends on the norm you had on vectors.