For my class, I am trying to understand why the norm of a matrix, is bounded in such a way. Specifically, I am trying to understand the rationale behind this equation. We are given the jacobian matrix $\frac{\delta{\mathbf{h}_i}}{\delta{\mathbf{h}_{i-1}}}$ being defined as:
$$ \frac{\delta{\mathbf{h}_i}}{\delta{\mathbf{h}_{i-1}}} = \prod_{i = k + 1}^{t}\mathbf{\theta}^T \text{diag}[\mathbf{\phi'}(\mathbf{h}_{i-1})] $$
Now given this, we are told that the (2-norm?) of the above jacobian is bounded as so:
$$ \lVert{\frac{\delta{\mathbf{h}_i}}{\delta{\mathbf{h}_{i-1}}}} \lVert \leq \lVert \mathbf{\theta}^T \lVert \lVert \text{diag}[\mathbf{\phi'}(\mathbf{h}_{i-1})]\lVert $$
Why is this true?
For a broader context, it is taken from those slides.
I can see and understand why the jacobian matrix $\frac{\delta{\mathbf{h}_i}}{\delta{\mathbf{h}_{i-1}}}$ is the way it is, but what I do not understand is where this equation bounding it's (I'm guessing 2-norm) comes from.
Thanks.
From the previous line of the slide,
$\frac{\partial h_{i}}{\partial h_{i-1}}=\Theta^{T}\mbox{diag}[\phi'(h_{i-1}]$
A basic property of matrix norms is that for any matrices $A$ and $B$ of compatible sizes,
$\| AB \| \leq \| A \| \| B \|$
Thus
$\left\| \frac{\partial h_{i}}{\partial h_{i-1}} \right\| \leq \| \Theta^{T} \| \| \mbox{diag}[\phi'(h_{i-1}] \|$