How to use Lipschitz continuity and Cauchy-Schwarz inequality to prove $(∇f(x) − ∇f(y))^T(x − y) ≤ L||x-y||^2$

Question

I have seen in a lot of texts that the ''Lipschitz continuity of $∇f$ and Cauchy-Schwarz inequality imply $(∇f(x) − ∇f(y))^T(x − y) ≤ L||x-y||^2$'' and I was wondering how to prove this using Lipschitz and the Cauchy Inequality?

Answer 1

If $L$ is a Lipschitz constant for $\nabla f$, then $$ \begin{split} (\nabla f(x) − \nabla f(y))^T(x − y) &=\bigl\langle \nabla f(x) − \nabla f(y),x-y\bigr\rangle \\ &\le \|\nabla f(x) − \nabla f(y)\|\cdot\|x-y\|\\ &\le L||x-y||^2. \end{split} $$ The first inequality comes from the Cauchy-Schwarz inequality.

Answer 2

We have Lipschitz continuity stating that:

$||∇f(x) − ∇f(y)|| \leq L||x-y||$

and Cauchy-Schwartz stating that:

$ u^{T}v= u \cdot v \leq ||u||*||v||$

thus,

$(∇f(x) − ∇f(y))^{T}(x - y) \leq ||(∇f(x) − ∇f(y))|| * ||x - y||$ or

$(∇f(x) − ∇f(y))^{T}(x - y) \leq L||x - y|| * ||x - y|| = L||x-y||^{2}$

Hence proved.

Answer 3

Cauchy-Schwartz inequality implies that $$(\nabla f(x) - \nabla f(y))^T (x-y) \leq \|\nabla f(x) - \nabla f(y)\| \|x-y\|.$$ Since the gradient is Lipschitz, we further have $$\|\nabla f(x) - \nabla f(y)\| \leq L \| x-y \|,$$ where $L$ is the Lipschitz constant. Combining these gives the result.