Thank you in advance for helping out an apsiring mathematician! :)
I am currently self-studying matrix norms (in the context of machine learning but that is not necessary for my question), and I came across a blog post: https://r2rt.com/written-memories-understanding-deriving-and-extending-the-lstm.html#fnref8 that had the following chain of reasoning. For context, $\Delta s_t$ and $\Delta s_{t+1}$ are vectors $\in \mathbb{R}^{n}$, $[\phi'(c)]$ is a matrix $\in \mathbb{R}^{n \times n}$ and so is $W \in \mathbb{R}^{n \times n}$. I will also use $|v|$ to denote the euclidean norm of a vector and $||A||$ to denote the $2$-norm of a matrix; I don't think it matters which particular matrix norm it is.
The claim is that, if $$ \Delta s_{t+1} = [\phi'(c)]W\Delta s_t $$ holds, then clearly, $$ |\Delta s_{t+1}| = |[\phi'(c)]W\Delta s_t| $$ and if we apply the matrix 2-norm twice, then we will obtain the bound $$ |\Delta s_{t+1}| \leq ||[\phi'(c)]||\cdot||W||\cdot|\Delta s_t| $$
What I don't get is how we can apply the matrix $2$-norm since we are dealing with scalars here. I tried inserting the identity matrix and using the properties of norms, but to no avail. I also tried working backwards from the result, where I obtain that $$ ||[\phi'(c)] W |\Delta s_t||| \leq ||[\phi'(c)]||\cdot||W||\cdot|\Delta s_t| $$ but I couldn't progress further.
Any explanation about why the claim is correct would be really appreciated, as I have spent quite a lot of time on this, and I can't figure out how to progress. Thank you in advance!