Showing Global Convergence of Damped Newton's Method

374 Views Asked by At

Suppose $f\in C^2(\mathbb{R}^n)$ satisfying $LI\succeq \nabla^2f(x) \succeq mI$ for all $x\in\mathbb{R}^n$ for some $L>m>0$. Let $x^*$ be the unique minimizer of $f$. Also suppose that, given the sequence $\{x_k\}_{k\geq 0}$ generated by damped Newton iteration,

\begin{align*} f(x)-f(x^*) \geq& \frac{m}{2}||x-x^*||^2 \quad \forall x\in\mathbb{R}^n \\ f(x_k)-f(x_{k+1}) \geq& \frac{m}{2L}\nabla f(x_k)^t (\nabla^2f(x_k))^{-1}\nabla f(x_k) . \end{align*} How would one show that $x_k\rightarrow x^*$ as $k\rightarrow\infty?$

Edit: I recently figured out how to prove the convergence, and thought I would put it here for others who were wondering the same.

By the non-negativity of the right hand side of the second inequality, we know that the sequence $\{f(x_k)\}$ is non-increasing. Also, $f$ is bounded below by $f(x^*)$. By the monotone convergence theorem, $f(x_k)$ must converge to something. So $f(x_k)-f(x_{k+1}) \rightarrow 0$. This implies that $\nabla f(x_k) \rightarrow 0$ since the Hessian is positive definite. Since $x^*$ is the unique minimizer, this implies that $f(x_k)\rightarrow f(x^*)$. By the first inequality, this implies that $x_k\rightarrow x^*$. $$\tag*{$\blacksquare$}$$