How is Lipschitz continuity for Fréchet derivatives defined?

Question

Let $(X,||\cdot||_X)$ be a Banach space and $X^*$ it's dual of linear functionals $X\to\mathbb{R}$. The Fréchet derivative $\nabla f(x)$ of a function $f:X\to\mathbb{R}$ at $x$ is an element of $X^*$.

What exactly does "Lipschitz-continuity of the gradient" mean in this context?

Is it $\forall x,y \in X$ $$ || \nabla f(x) - \nabla f(y)||_{X^*} \leq L || x -y ||_{X} $$ where $|| g ||_{X^*} = \sup \{ |g(x)| : x \in X, ||x|| \leq 1 \}$ is the dual norm and $L$ the Lipschitz constant or

$$ |\langle \nabla f(z), x-y \rangle| \leq L || x -y ||_{X} $$ where $\langle \cdot, \cdot \rangle$ is the duality map?

If the latter is correct, does it have to hold for all $z \in X$?

Edit: An example that would help me to understand this issue better would be $$ f: x \mapsto \frac{1}{2}||x-c||_X^2 $$ for some constant $c \in X$, with derivative $Df(x): h \mapsto \langle x-c,h \rangle$ and gradient $\nabla f(x) = x-c$. How can I calculate the Lipschitz constant?

Answer 1

The Fréchet derivative $\nabla f(x)$ of a function $f:X\to\mathbb{R}$ at $x$ is an element of $X^*$.

And therefore, $\nabla f$ is a map from $X$ to $X^*$. The Lipschitz condition makes sense whenever there is a map between metric spaces; and both $X$ and $X^*$ have metrics induced by their norms. Explicitly, $$\| \nabla f(x) - \nabla f(y)\|_{X^*} \leq L \| x -y \|_{X}$$

Your example $$f(x) = \frac{1}{2}\|x-c\|_X^2$$ is very interesting. In a general Banach space, $\nabla f$ is not even defined: for example, the $\ell_1$ norm is not differentiable when one of coordinates is zero (consider two-dimensional model to make this very explicit).

A space for which $f$ is Fréchet differentiable is called a space with Fréchet differentiable norm (the norm itself is of course not differentiable at $0$, but the term refers to differentiability at nonzero points). In lecture 23 here is is proved that in such a space, $\nabla f$ is automatically continuous. The map $\nabla f$ is called the support map or the duality map: it sends each element $x$ to a linear functional $\varphi $ such that $\|\varphi\|=\|x\|$ and $\|\varphi(x)\|=\|x\|^2$.

In general, $\nabla f$ is not even uniformly continuous, let alone Lipschitz continuous.

• $\nabla f$ is uniformly continuous $\iff$ $X$ is a uniformly smooth space
• $\nabla f$ is Lipschitz continuous $\iff$ $X$ is a uniformly smooth space with quadratic modulus of smoothness.

The Lipschitz constant of $\nabla f$ depends on the multiplicative constant in the modulus of smoothness.