Matrix Multiplication Norm

298 Views Asked by At

I'm trying to prove a proposed theorem for my thesis and was wondering if the following property is true and can be used in my proof.

$ \bf {s ^T M sgn (s)} < \parallel \bf s \parallel \parallel M \parallel $

where the function $\bf {sgn (s)}$ is the signum (sign) function, the dimensions of the vector $\bf s$ and the matrix $\bf M$ is such that the product $ \bf {s ^T M sgn (s)}$ is a scalar and the norm operator represent the euclidian norm.

1

There are 1 best solutions below

0
On

Suppose that we define $\mathrm{sign}(s)$ elementwise, as follows:

$$ s = \begin{pmatrix} s_1 \\ \vdots \\ s_n \end{pmatrix} \Rightarrow \mathrm{sign}(s) = \begin{pmatrix} \mathrm{sign}(s_1) \\ \vdots \\ \mathrm{sign}(s_n)\end{pmatrix} $$

where we denote $$ \mathrm{sign}(z) = \begin{cases} 1, & z \geq 0 \\ -1, & z < 0 \end{cases} $$

For that case, notice that

$$ s^\top M \mathrm{sign}(s) = \mathrm{trace}(s^\top M \mathrm{sign}(s)) = \mathrm{trace}(\mathrm{sign}(s)s^\top M) \quad (\text{cyclic invariance of trace}) \\ \leq \sqrt{\mathrm{trace}\left((\mathrm{sign}(s)s^\top)^\top \mathrm{sign}(s)s^\top\right)} \cdot \sqrt{\mathrm{trace}(M^\top M)} \quad (\text{Cauchy-Schwarz}) $$

Now, notice that the latter quantity is exactly $\left\| M \right\|_2$, and that

$$ \mathrm{trace}\left( s \cdot \mathrm{sign}^\top(s) \mathrm{sign}(s) s^\top) \right) = n s^\top s = n \left\| s \right\|^2 $$

since the inner product of the sign vectors is $$ \sum_{i=1}^n [\mathrm{sign}(s_i)]^2 = n \cdot 1 $$

The above imply that the following inequality holds:

$$ s^\top M \mathrm{sign}(s) \leq \sqrt{n} \left\| s \right\| \left\| M \right\| $$

Therefore, you can prove your claim if you augment $\mathrm{sign}(s)$ with an appropriate normalization constant.