Is the spectral norm of the Jacobian of an $M$-Lipschitz function bounded by $M$?

Question

Well, the title pretty much says everything. I have a function $f: \mathbb{R}^n \mapsto \mathbb{R}^n$, which is $M-$Lipschitz with respect to the vector $L^2$ norm, i.e. $$||f(x)-f(y)||_2\leq M ||x-y||_2~\forall~x,y \in \mathbb{R}^n~.$$ Let $J_f: \mathbb{R}^n \mapsto \mathbb{R}^{n\times n}$ denote the Jacobian function of $f$, i.e. $$\left(J_f(x)\right)_{i,j} = \frac{\partial}{\partial x_j} f_i(x)~.$$ Assume that $J_f(x)$ is symmetric for every $x$. My question is, is the following inequality true (and why, if so)? $$||J_f(x)||_2 \leq M~\forall~x \in \mathbb{R}^n~,$$ where the last norm $||\cdot||_2$ refers to the spectral norm, i.e. the largest absolute singular value.

Answer 1

$$\newcommand{\a}{\mathbf{a}} \newcommand{\b}{\mathbf{b}} \newcommand{\x}{\mathbf{x}} \newcommand{\p}{\mathbf{p}}$$ Assume that $f: \Omega \rightarrow \mathbb{R}^m$ where $\Omega \subseteq \mathbb{R}^n$ and that $f$ is differentiable on $\Omega$ with the Jacobian $J_f(\mathbf{x})$ at $\mathbf{x} \in \Omega$. A consequence of differentiability is that the directional derivative of $f$ at $\x$ in the direction of an arbitrary unitary norm vector $\p$ is $J_f(\x)\p$. More formally, for any unitary norm $\p$ and $\epsilon>0$ we can find $\delta > 0$ such that if $h<\delta$ we have: \begin{equation} \frac{\|f(\x+h\p)-f(\x) - J_f(\x)\p h\|}{h} < \epsilon \end{equation} Applying the triangle inequality $\big| \|\a\| - \|\b\| \big| \leq \|\a-\b\|$ where $\a = f(\x + h\p)-f(\x)$ and $\b = J_f(\x)\p h$ , we obtain \begin{equation} \left|\frac{\|f(\textbf{x}+h\textbf{p})-f(\textbf{x})\|}{h} - \|J_f(\textbf{x})\textbf{p}\|\right| < \epsilon \end{equation} Hence: \begin{equation} \|J_f(\textbf{x})\textbf{p}\| < \frac{\|f(\textbf{x}+h\textbf{p})-f(\textbf{x})\|}{h} + \epsilon < M + \epsilon \end{equation} where $M$ is the Lipschitz constant. Since this must hold for every $\epsilon$, we have that: \begin{equation} \|J_f(\textbf{x})\textbf{p}\| < M \end{equation} Since by definition of the spectral norm: \begin{equation} \|J_f(x)\| = \sup_{\|p\|=1} \|J_f(\mathbf{x})\mathbf{p}\| \end{equation} we have that $\|J_f(x)\| \le M$