Proving convergence of Newton-Raphson using contraction mapping theorem

318 Views Asked by At

I'm trying to prove there exists an $\epsilon \in \mathbb{R}$ such that for a root, $p$, of a function $f \in C^2$, we have that $\forall q \in (p - \epsilon, p + \epsilon)$, $q$ converges to $p$ due to Newton-Raphson iteration. We are given that $f(p) = 0, f'(p) \neq 0$. Most proofs online prove quadratic convergence using Taylor series; I am not looking for a quadratic bound, but just convergence generally, and I specifically want to use the contraction mapping (Banach fixed-point) theorem.

My attempt is as follows. First, I see it suffices to show that $g: x \mapsto x - \frac{f(x)}{f'(x)}$ is a contraction, since fixed points of $g$ are zeros of $f$ by construction. Then the result follows because we simply need to find an interval around $p$ where $f'(x) \neq 0$, which we can do since we're looking for an open interval and by definition of open, one exists. Call it $U$. And so we come down to trying to show $g$ is a contraction. First, by MVT we have that $g(x) - g(y) = g'(k)(x-y)$ for some $k \in (x,y)$ where $x, y \in U$. And so it suffices to show that $|g'(k)| < 1$ for it to be our contraction constant. We differentiate $g(x)$ to get $g'(x) = \frac{f(x)f''(x)}{f'(x)^2} \implies g'(k) = \frac{f(k)f''(k)}{f'(k)^2}$. I am not sure how to then prove that the magnitude of this term is strictly less than one. Could someone comment on if my general approach is correct, and if so, how I should show that this term is sufficiently small in size to conclude the proof?

1

There are 1 best solutions below

1
On BEST ANSWER

Generally I think this approach is correct and interesting. Having not taken too many numerical analysis or applied courses myself, I have never seen a proof of NR convergence using Banach's FP Theorem so I think this is rather neat.


Before we discuss finishing the proof, I just want to make a minor comment and you probably already understood this but it wasn't entirely clear from your answer: you said at one point that

we simply need to find an interval around $$ where $′()≠0$, which we can do since we're looking for an open interval and by definition of open, one exists.

It is true that we can find an open interval around $p$ such that $f'(x) \neq 0$ but this is because of the continuity of $f'$ at $p$ (given that $f\in C^2(\mathbb{R})$) and not, as far as I can see, "by the definition of open" at all. For example, if we took some $f\not\in C(\mathbb{R})$ such as $f:\mathbb{R}\to\mathbb{R}$ defined by $f(x) = x+x^2\sin\left(\frac{1}{x}\right)$ then this function:

  • is differentiable on all of $\mathbb{R}$ (left as an exercise in limits or can be seen by Caratheodory's Theorem)
  • has $f'(0) = 1$
  • for nonzero $x$, has $f'(x) = 1+2x\sin\left(\frac{1}{x}\right) - \cos \left(\frac{1}{x}\right)$, which is continuous everywhere but $x = 0$ and vanishes on every point of the form $x = \frac{1}{2\pi n}$ for $n\in\mathbb{N}$.

Thus this function has no open interval around the origin where $f'(x)\neq 0$.


Now onto the proof. We want an interval $I \subseteq U$ such that $|g'(k)| = \left|\frac{f(k)f''(k)}{f'(k)^2}\right|<1$ for $k\in I$.

Note that we haven't so far in your proof used the fact that $f\in C^2(\mathbb{R})$, only that $f\in C(\mathbb{R})$, which should be ringing alarm bells. We know that as $k\to p$ that $f(k)\to 0$ since $f$ is continuous at $p$ and $f(p) = 0$ by choice so all we need to show is that the other terms in the quotient don't "cause any problems". But this isn't so difficult: by $f\in C^2(\mathbb{R})$ we know that both $f', f''$ are continuous on $U$ and additionally we know that $f'$ is nonzero on this open interval so $\frac{f''}{f'}$ is a continuous function on $U$. Given this, we know that on any closed interval subinterval of $U$ this quotient is bounded on $U$ (by the boundedness theorem), i.e. there is some $M>0$ such that for all $x\in U$ we have $\left|\frac{f''(x)}{f'(x)}\right| < M$.

Then we are essentially done: since $f(k)\to 0$ as $k\to p$ we know that there is some $\varepsilon>0$ such that $x\in (p-\varepsilon, p+\varepsilon)\implies |f(x)| < \frac{1}{M}$ and if we choose $\varepsilon$ sufficiently small such that $(p-\varepsilon, p+\varepsilon) \subseteq U$ then we have for $x\in (p-\varepsilon, p+\varepsilon)$ that $$|g'(x)| = |f(x)|\cdot\left|\frac{f''(x)}{f'(x)}\right| < \frac{1}{M}\cdot M = 1$$ so this $\varepsilon$ meets your requirements.