Relation between matrix norms and contractions

115 Views Asked by At

I'm not so familiar with matrix norms, so I attempted an elementary proof to find conditions on the map $\phi(x) = Ax + b$ to be a contraction under the metrics induced by the $L^1$ and $L^2$ norms. Here, $x, b \in \mathbb{R}^d$ and $A \in \mathbb{R}^{d \times d}$. Clearly, under any of the mentioned metrics $d$ we have that $d(\phi(x), \phi(y)) = d (Ax, Ay)$, so $b$ can be any vector and it does not affect the fact that $\phi$ might or not might be a contraction, this fact only depends on $A = (a_{kl})$.

I'll denote the metric induced by the $L^p$ norm as $d_p$.

For $d_1$ I found the following, for any $x, y \in \mathbb{R}^d$:

$$d_1(Ax, Ay) = \sum_{k = 1}^d \left \vert (Ax)_k - (Ay)_k \right \vert = \sum_{k = 1}^d \left \vert \sum_{l = 1}^d a_{kl} (x_l - y_l) \right \vert.$$

Now, using the triangle inequality, this is $\leq$ than:

$$\sum_{k = 1}^d \sum_{l = 1}^d \left \vert a_{kl} (x_l - y_l) \right \vert = \sum_{l = 1}^d \vert x_l - y_l \vert \sum_{k = 1}^d \vert a_{kl} \vert.$$

Assuming that $0\leq \displaystyle \sum_{k = 1}^d \vert a_{kl} \vert \leq \max_{1 \leq l \leq d} \sum_{k = 1}^d \vert a_{kl} \vert = \vert \vert A \vert \vert_1 \leq a < 1$, we now get that:

$$\sum_{l = 1}^d \vert x_l - y_l \vert \sum_{k = 1}^d \vert a_{kl} \vert \leq a d_1(x, y) < d_1(x, y).$$

So a sufficient condition on $A$ for $\phi$ to be a contraction under the $L^1$ metric is that it's 1-norm is strictly bounded above by 1.

In essence what I proved is that $\vert \vert Ax \vert \vert_1 \leq \vert \vert A \vert \vert_1 \vert \vert x \vert \vert_1$ for any $x \in \mathbb{R}^d$.

Now, for $L^2$ I did something similar, but my result is less symmetrical than the one I just found.

Again, for any $x, y \in \mathbb{R}^d$,

$$d_2(Ax, Ay)^2 = \sum_{k = 1}^d \left \vert (Ax)_k - (Ay)_k \right \vert^2 = \sum_{k = 1}^d \left \vert \sum_{l = 1}^d a_{kl} (x_l - y_l) \right \vert^2.$$

By Cauchy-Schwarz, this is $\leq$ than:

$$\sum_{k = 1}^d \left( \sum_{l = 1}^d a_{kl}^2 \right) \left( \sum_{l = 1}^d (x_l - y_l)^2 \right) = d_2(x, y)^2 \sum_{k = 1}^d \sum_{l = 1}^d a_{kl}^2 = \vert \vert A \vert \vert_F^2 d_2(x, y)^2.$$

Where $\vert \vert A \vert \vert_F$ is the Frobenius norm of $A$. If this value is less than 1, we get that $d_2(\phi(x), \phi(y))^2 < d_2(x, y)^2$ which implies that $\phi$ is a contraction under this norm.

Now, my question is, why didn't a $L^2$ norm appear on the matrix?

Is it also true that if $\vert \vert A \vert \vert_2 < 1$, then $\phi$ is a contraction under the $L^2$ metric?

And, is in general true that $\phi$ is a contraction under the $L^p$ metric (for $1 \leq p \leq \infty$ if $\vert \vert A \vert \vert_p < 1$?

1

There are 1 best solutions below

0
On

I've spent the whole day reading about matrix norms and I solved my issue.

Under any norm $\vert \vert \cdot \vert \vert$ on $\mathbb{R}^d$ there's an induced matrix norm given by:

$$\vert \vert A \vert \vert = \max_{x \neq 0} \frac{\vert \vert A x \vert \vert}{\vert \vert x \vert \vert}.$$

So it's easy to prove that $\vert \vert Ax \vert \vert \leq \vert \vert A \vert \vert \cdot \vert \vert x \vert \vert$ since:

$$\vert \vert A \vert \vert \cdot \vert \vert x \vert \vert = \vert \vert x \vert \vert \max_{y \neq 0} \frac{\vert \vert A y \vert \vert}{\vert \vert y \vert \vert} \geq \vert \vert x \vert \vert \frac{\vert \vert A x \vert \vert}{\vert \vert x \vert \vert} = \vert \vert A x \vert \vert.$$

Now, for any $p \geq 1$, we have that:

$$d_p(\phi(x), \phi(y))^p = \vert \vert A x + b - A y - b \vert \vert_p = \vert \vert A (x - y) \vert \vert_p \leq \vert \vert A \vert \vert_p \vert \vert x - y \vert \vert_p < \vert \vert x - y \vert \vert_p = d_p(x, y)^p.$$

Where the last strict inequality involves the hypothesis of $\vert \vert A \vert \vert_p < 1$. In these cases, the map $\phi$ is a contraction.

So for my particular case, in the $L^2$ metric, it's more than sufficient to have $\vert \vert A \vert \vert_F < 1$, with only $\vert \vert A \vert \vert_2 < 1$ we get that our map is a contraction; and since this norm is just the max singular value of $A$ it's even better!