How does a matrix change the magnitude of a vector?

Question

I have the following problem:

$z=Ax$, in which $z$ and $x$ are $N\times 1$ vectors and $A$ is a $N\times N$ matrix. I am interested in how the magnitude of $x$ changes after applies $A$ on it. Is there any simple relationship between the properties of $A$ (e.g., eigenvalues) and the differences in magnitudes between $z$ and $x$?

Thanks!

Answer 1

Supposing you put yourself in an orthonormal basis of eigenvectors (supposing it exists), you have $$ z_i=(Ax)_i=\lambda_i x_i $$ so that $$ |z|^2=\sum_{i=1}^N z_i^2=\sum_{i=i}^N \lambda_i^2 x_i^2 $$ and this cannot be expressed as a function of $|x|$ (unless $N=1$).
In the particular case in which $x$ is an eigenvector, then the sum reduces to a single term and the magnitude change by a factor $|\lambda_i|$.

If we define $$ m=\min{|\lambda_i|}\\ M=\max{|\lambda_i|} $$ then one can say that $$ |z|^2=\sum_{i=i}^N \lambda_i^2 x_i^2\leq M^2 \sum_{i=i}^N x_i^2 = M^2 |x|^2\\ |z|^2=\sum_{i=i}^N \lambda_i^2 x_i^2\geq m^2 \sum_{i=i}^N x_i^2 = m^2 |x|^2 $$ so that $$ m|x|\leq |z|\leq M|x|. $$

Answer 2

An operator (matrix) $A$ is said to be bounded with respect to norm $|| \cdot ||$ if there exists a (finite) constant $C$ such that for all $x$, $||Ax|| \leq C ||x||$. The smallest such $C$ is called the operator norm of $A$ and is denoted $||A||$.

It depends on the norm you consider on $x$ to measure magnitude, but for the 2-norm, the largest singular value of A, $||A||_2=\sigma_\max$, satisfies $||A x||_2 \leq \sigma_{\max} ||x||_2$.

(That is to say, the operator norm of the matrix $A$ with respect to the 2-norm is the largest singular value of $A$)

If you consider the 1-norm, $||A||_1$ is the largest column sum of $A$ while $||A||_\infty$ is the largest row sum of $A$.

Then, you have $||A x||_1 \leq ||A||_1 ||x||_1$ and $||A x||_\infty \leq ||A||_\infty ||x||_\infty$.