Concerning the sequence of gradients in Nesterov's Accelerated Descent

63 Views Asked by At

Suppose we apply Nesterov's accelerated gradient descent to a $L$-smooth convex function $f \colon \mathbb R^n \to \mathbb R \cup \{\infty\}$, \begin{align*} & x_k = y_k - \frac{1} {L} \nabla f(y_k) \\ & y_{k+1}= x_k + \frac{k+1} {k+3} (x_k - x_{k-1}). \end{align*}

We know the function values $\{f(x_k)\}$ might not monotonically decrease.

What about the sequence of $\{\|\nabla f(x_k)\|_2\}$? Are they monotonic? More specifically, I am interested to know if we assume the minimum $x^*$ exists, will the seqence of gradients $\{\|\nabla f(x_k)\|_2^2\}$ generated be bounded?