Inequality for convex function say $f$ with $L$-Lipschitz continuous gradient: $( x - y)^T \left( \alpha \nabla f(x) - \beta \nabla f(y)\right)$?

145 Views Asked by At

Any known known bound for a convex function say $f$ with $L$-Lipschitz continuous gradient (for $\alpha, \beta \in \mathbb{R}$, which can be $\alpha \neq \beta$):

  • $( x - y)^T \left( {\color{red} \alpha} \nabla f(x) - {\color{red} \beta} \nabla f(y)\right) \leq \ \color{red} ? $
  • $\left\| {\color{red} \alpha} \nabla f(x) - {\color{red} \beta} \nabla f(y)\right\| \leq \ \color{red} ? $

(Related to my previous question, but not really.)

1

There are 1 best solutions below

3
On BEST ANSWER

I am not sure whether this is the tightest bound which you can achieve. But here is my attempt for part 1:

Let's take the case $\alpha,\beta >0$ and $\alpha \geq \beta$

\begin{align} ( x - y)^T \left( { \alpha} \nabla f(x) - {\beta} \nabla f(y)\right) &= \frac{(\alpha + \beta)}{2}(x - y)^T(\nabla f(x) - \nabla f(y)) \ + \frac{(\alpha - \beta)}{2}(x-y)^T(\nabla f(x) + \nabla f(y)) \\ &\leq \frac{(\alpha+\beta)}{2}L||x-y||^2 + \frac{(\alpha - \beta)}{2}(x-y)^T(\nabla f(x) + \nabla f(y)) \end{align} The second term can grow unbounded in general case unless $f$ is Lipschitz continous. Assuming $f$ is $G$-Lipshitz continuous the bound becomes then we have

$$ |\nabla f(x)| \leq G $$ Then by Cauchy-Scwarz inequality we have: \begin{align} ( x - y)^T \left( { \alpha} \nabla f(x) - {\beta} \nabla f(y)\right) &\leq \frac{(\alpha+\beta)}{2}L||x-y||^2 + \frac{(\alpha - \beta)}{2}*(2G||x-y||) \end{align}