How are Taylor's Theorem and Cauchy Schwartz inequality used here?

Question

I have been trying to get my head around this proof, but i just don't manage to do so.

I am ok with the first two lines, it is just the definition.

But on the next line, how is Cauchy-Schwartz inequality exaclty applied?

And on the next line, I also don't understand how is Taylor's Theorem used.

Update: I just managed to figure out the usage of Cauchy-Schwartz. But I still don't understand how Taylor's Theorem was used.

Answer 1

Applying the L-smooth inequality to $y = x + \xi (y -x)$ we get $$ \| \nabla f(x + \xi (y -x)) - \nabla f(x) \| \leq L \xi \| y - x\| $$

Using Cauchy-Schwarz you know: $$ | (\nabla f(x + \xi (y -x)) - \nabla f(x)) \cdot (y - x) | \leq \| (\nabla f(x + \xi (y -x)) - \nabla f(x)) \| \| (y - x) \| $$ and using the previous inequality you get: $$ \| \nabla f(x + \xi (y -x)) - \nabla f(x) \|\ \| y - x \| \leq L \xi \| y - x\|^2 $$

About the usage of Taylor theorem, I believe the integral is the remainder in integral form for the first order Taylor series.