$\|f'''(x)[u]\| \leq M\|u\|$ implies $M$-Lipschitz continuity of the Hessian

81 Views Asked by At

Let $f: \mathbb{R}^{n} \rightarrow \mathbb{R}$ be a $C^{3}$ function and let $\nabla^{2}f(x)$ be the Hessian of $f$ at $x$. Furthermore, let

$$f'''(x)(u) = \lim_{\alpha \to 0} \frac{1}{\alpha}(\nabla^{2}f(x+\alpha{u})-\nabla^{2}f(x)).$$

I want to show that if $$\|f'''(x)(u) \| \leq M \|u\|$$ then the Hessian of $f$ is $M$-Lipschitz continuous: $$\|\nabla^{2}f(x) - \nabla^{2}f(y) \| \leq M \|x-y \|.$$

I should note that $f'''(x)(u)$ is a $n \times n$ matrix and I'm interpreting $\|f'''(x)(u) \|$ as the operator norm, though I am not sure if this is the correct interpretation.

For context, this fact is used in the motivation for self-concordant functions. The statement that I m trying to prove appears in pg 329 of Nesterov's Lectures of Convex Optimization.

1

There are 1 best solutions below

0
On BEST ANSWER

Note that $f'''(x)(u)$ is simply the directional derivative of $\nabla^2 f$ at $x$ in the direction $u$. Hence, with $u=y-x$, the fundamental theorem of calculus shows that $$||\nabla^2 f(x)-\nabla^2 f(y)|| = \left\lVert \int_0^1 f'''(x +tu)(u)\,dt\right\rVert \\ \leq \int_0^1 \left\lVert f'''(x+tu)(u)\right\rVert dt \\ \leq M ||u||.$$

The first inequality is the triangle inequality for general norms (note that the integrand is matrix-valued), which holds for any norm.