I would like to know the theoretical number of iterations for the BFGS method to minimise a quadratic function.
From this source: http://www.aptech.com/wp-content/uploads/2013/05/qnewton.pdf
On page 2 line 4 it says that we only need a single step to solve a quadratic equation.
From this source: http://www.bioinfo.org.cn/~wangchao/maa/Numerical_Optimization.pdf
p210 theorem 8.4 it says that: (i) The iterates converge to the solution in at most n iterations.
So which is it? If anyone could offer some clarity that would be great.
On page 2 line 4 they are referring to Newton's method, not BFGS.
Newton's method minimizes a quadratic function in one iteration. Quasi-Newton methods do not.