How $||\nabla f(x) - \nabla f(\overline{x})||\le L|x-\overline{x}|$ implies $(\nabla f(x) -\nabla f(\overline{x}))^Tp_k\le L|x-\overline{x}||p_k|$

179 Views Asked by At

I'm trying to understand how the lipschitz condition (3.13) implies the last equation. The rest is specific for optimization and is not needed to understand the question but I leave it here for context. The book is from Nocedal.

enter image description here

As far as I understood, the Lipstichtz condition is on the norm. We can only say what's in 3.13. However, the least equation of the image is not on the norm. How do I prove such implication? (the right side is just $L||x_k+\alpha_k p_k - x_k||$)

Not necessary for the question but just to inform:

If you want to know more about it, the conditions 3.6b and 3.1 are, respectively:

$$\nabla f_{k+1}^Tp_k\ge c_2\nabla f_k^Tp_k$$ $$x_{k+1} = x_k + \alpha_k p_k$$

1

There are 1 best solutions below

0
On BEST ANSWER

\begin{align} (\nabla f_{k+1}-\nabla f_k)^T p_k &\le \|\nabla f_{k+1}-\nabla f_k\|\|p_k\|, \text{by Cauchy-Schwarz}\\ &\le L \|x_{k+1}-x_k\|\|p_k\| \text{, by Lipschitz} \\ &=L\|\alpha_k p_k\|\|p_k\|\\ &= L\alpha_k \|p_k\|^2 \end{align}