Show that if $f $ is $L$-smooth, then $g(x) := f(x) - \frac{m}{2} \Vert x \Vert^2$ is $(L-m)$-smooth

701 Views Asked by At

A continuously differentiable function $f : \mathbb{R}^n \rightarrow \mathbb{R}$ is $L$-smooth if $\nabla f$ is $L$-Lipschitz, i.e., for all $x,y \in \mathbf{dom}\,f$, $$ \Vert \nabla f(x) - \nabla f(y) \Vert \leq L\Vert x- y \Vert. $$

Define $g(x) := f(x) - \frac{m}{2} \Vert x \Vert^2$ for some constant $m \geq 0$, and I need to show that $g$ is $(L-m)$-smooth. It seems like a simple enough proof but I keep arriving at an inconclusive result using the triangle inequality: rearrange the equation and we can get $\nabla f(x) = \nabla g(x) + mx$; by substitution, we have $$ \begin{align} \Vert \nabla f(x) - \nabla f(y) \Vert &= \Vert \nabla g(x) + mx - \nabla g(y) - my \Vert \\ &= \Vert \nabla g(x) - \nabla g(y) + m(x-y) \Vert \\ &\leq \Vert \nabla g(x) - \nabla g(y)\Vert + m\Vert x-y \Vert, \\ \end{align} $$ but we don't necessarily have $\Vert \nabla g(x) - \nabla g(y)\Vert + m\Vert x-y \Vert \leq L\Vert x- y \Vert$ to finish the proof.

What am I missing here? Any hint is greatly appreciated. The original statement is in the proof of Proposition 5, p. 13.

Update
It is actually required that $0 < m < L$, but I didn't change this in the original question.

1

There are 1 best solutions below

0
On

As observed $f$ should be m-strongly convex. We start observing that $g(x)=f(x) - \frac{m}{2} \lVert x \rVert^2$ is convex and defined over $\mathbb R^n$. In this case, it is enough to prove the quadratic upper-bound:

$$g(y) \le g(x) + \nabla g(x)^\intercal (y-x) + \frac{1}{2}(L-m) \lVert y-x \rVert^2$$ (for the equivalence see for example Nesterov, Lectures on Convex Optimization, Theorem 2.1.5).

This relation follows from replacing the definition of $g$ in the following inequality for $f$ ($f$ is L-smooth): $$f(y) \le f(x) + \nabla f(x)^\intercal (y-x) + \frac{1}{2}L \lVert y-x \rVert^2.$$