Why Newton's method to find a root of a function requires the function to be smooth?

Question

I was reading a chapter of a book which says that Newton's method, as a general purpose algorithm for finding zeros of functions, it has three serious drawbacks:

1. The function $f(x)$ must be smooth

2. It might not be convenient to compute the derivative o $f'(x)$.

3. The starting guess must be close to the final result

I understand the second point very well and roughly the third one. Of course since Newton's method uses the derivative in its iterations, finding the derivative should be as easy as possible to apply this method. For the third, it seems clear the statement, but I would not know exactly how to explain it.

For the first point instead, I've no idea why it is required, except that if a function is not smooth, at some points the derivative may not exist, am I right?

Why Newton's method to find a root of a function requires the function to be smooth?

Answer 1

Smoothness is an overused term in mathematics. It's not always clear what the author means when a quote is taken out of context. Most often, smooth describes a function whose derivatives of all orders exist. Sometimes, though, it can mean just once differentiable. I assume this is the meaning taken here.

We can use Newton's method for functions which at least once differentiable. Since the method requires the calculation of the slope of a tangent line, the first derivative of the function must exist. If not, the algorithm may lead us to attempt to differentiate the function at a point where the derivative doesn't exist, and then what do we do? Newton's algorithm would fail in this case.

Answer 2

Newton's method arising from considering the Taylor series expansion for a function $f$ and then discarding the higher-order terms and solving. If your textbook is using Taylor series to motivate Newton's method then it will require that $f$ be smooth because the Taylor series needs that.

However, to apply Newton's method in general you only need first order differentiability at all points of the domain so that your iterative step exists and can be calculated. Smoothness won't hurt, but it's a little bit of overkill.

To see how Newton's method arises, consider that a smooth (infinitely differentiable) function $f$ has the Taylor series expansion about a point $x_0$ given by: $$f(x_0 +h) = f(x_0) + f'(x_0)\cdot h + f''(x_0)\cdot h^2 + \cdots$$ Discarding terms above first order to obtain an approximation we get $$f(x_0 +h) \approx f(x_0) + f'(x_0)\cdot h$$ Setting $f(x_0+h) = 0$ (we're looking for a root and hoping that $x_0+h$ is it) and solving for $h$ we get: $$ h = \frac{-f(x_0)}{f'(x_0)}$$ Then if $x_1 = x_0+h$ and we want to iterate that we must have $$x_n = x_{n-1} - \frac{f(x_0)}{f'(x_0)}$$