Combining Nesterov Acceleration with Exact Line Search for Least Squares Problem

299 Views Asked by At

Nesterov accelerated gradient descent is one way to accelerate the gradient descent methods.

The exact line search is also one way to find the optimal step size along the gradient direction for the least-squares problems.

However, I have not seen anything related to the combination of Nesterov acceleration and exact line search.

Is there something that impedes the combination?

1

There are 1 best solutions below

0
On

You may have a look at A Field Guide to Forward Backward Splitting with a FASTA Implementation (Project Website).

They have some heuristics to combine both acceleration and line search.