Prove that superlinear convergence of gradients implies superlinear convergence of sequence itself

193 Views Asked by At

I'm stuck on proving a result that is not specifically proven in a research paper.

https://people.maths.ox.ac.uk/cartis/papers/ARCpI.pdf

It is the proof of Corollary 4.8. I'm trying to show that $(4.33)\implies (4.34)$.

The way I understand it, the statement is as follows:

Theorem:

Let $f: \mathbb{R}^n\to \mathbb{R}$, $f\in C^2$. Suppose $x_k\to x_*\in \mathbb{R}^n$, $\nabla f(x_*)=0$ and $\nabla^2f(x_*)\succ 0$ (positive definite). Assume $x_k \neq x_*$ and $\nabla f(x_k)\neq 0$ for every $k$, and also, $$\lim_{k\to\infty} {||\nabla f(x_{k+1})||\over ||\nabla f(x_k)||} = 0.$$

Prove that $$\lim_{k\to\infty} {||x_{k+1}-x_*||\over ||x_k-x_*||} = 0.$$

I know by Taylor's Theorem that

$$\nabla f(x_k) = \int_{0}^1 P(t)(x_k-x_*)dt$$

where $P(t)=\nabla^2 f(x_*+t(x_k-x_*))$.

Continuity gives us that $\nabla^2 f(x_*)^{-1}\nabla f(x_k) -(x_k-x_*)\to 0$ as $k\to\infty$.

Letting $H=\nabla^2 f(x_*), g_k=\nabla f(x_k)$, we have, for every $\epsilon>0$, for large enough $k$, that $$||H^{-1}g_k||\in (||x_k-x_*||-\epsilon, ||x_k-x_*||+\epsilon).$$

I am unsure how to proceed from here. Knowing that $g_k\to0, {||g_{k+1}||\over ||g_k||}\to0$, I can easily show that $H^{-1}g_k\to 0$ and ${||H^{-1}g_{k+1}||\over ||H^{-1}g_k||}\to0$. If I can show that the latter ratio is an upper bound of a positive constant times the ratio in my desired limit result, then I'm done. That is, if there is some $C>0$ such that , for $k$ large enough, $$C{||x_{k+1}-x_*||\over ||x_k-x_*||}\le {||H^{-1}g_{k+1}||\over ||H^{-1}g_k||}.$$

Am I using the right approach? Any help is greatly appreciated.

Please Note. Even though we have $0=\lim ||H^{-1}g_k|| = \lim ||x_k-x_*||$, the second equality with direct substitution does not suffice, as the ratio structure can be damaged.