Bounding an $L$-smooth function $f$: $(x-y)^T \left[ \nabla f(x) - \nabla f(z) \right] \geq ? $

216 Views Asked by At

Let $f$ be a convex function with $L$-Lipschitz continuous gradient.

One can upper bound $( x - y)^T \left[ \nabla f(x) - \nabla f(z)\right]$ by applying Young's inequality followed by Lipschitz continuous gradient condition, that is, \begin{align} ( x - y)^T \left[ \nabla f(x) - \nabla f(z)\right] &\leq \frac{1}{2} \left( \left\| x - y \right\|^2 + \underbrace{\left\| \nabla f(x) - \nabla f(z) \right\|^2}_{ \leq L^2 \left\| x - z \right\|^2 } \right) \\ &\leq \frac{1}{2} \left( \left\| x - y \right\|^2 + L^2 \left\| x - z \right\|^2 \right). \end{align}

How to lower bound the inequality in terms of sum and/or difference of norms between the differences among $x,y,z$ vectors:

  • $ ( x - y)^T \left[ \nabla f(x) - \nabla f(z)\right] \geq \ {\color{red} ?} , \quad \forall x,y,z$.

I would highly appreciate your help.


p.s.: Lipschitz continuous gradient conditions are summarized here https://xingyuzhou.org/blog/notes/Lipschitz-gradient.


EDIT:

Does the below bound not make sense considering Cauchy-Schwarz inequality? $ ( x - y)^T \left[ \nabla f(x) - \nabla f(z)\right] \geq \ { \color{green}{ -\frac{1}{2}\left( \left\| x - y \right\|^2 + L^2\left\| x - z \right\|^2 \right) }} , \quad \forall x,y,z$