Why does the error term in Taylor's Theorem converge to Zero (in two senses)?

Question

Let us consider Taylor's Theorem in the case of one real variable. For simplicity let us assume that $f$ is infinitely differentiable about $a$. Then we have that

$$f(x) = f(a) + f'(a)(x-a) + \frac{f''(a)}{2!}(x-a)^2 + \cdots + \frac{f^{(k)}(a)}{k!}(x-a)^k + h_k(x)(x-a)^k$$

My question is why does the remainder term converge to $0$? In particular, why do we have that

$$ \lim_{x \to a} h_k(x) = 0$$

as well as that

$$ \lim_{k \to\infty} h_k(x) = 0?$$

If I understood this last statement, in particular, then I would understand why

$$f(x) = \sum_{k=0}^{\infty} \frac{f^{(k)}(a)}{k!} (x-a)^k$$

which has also been a source of confusion for me.

Answer 1

There are two questions here. Firstly, you ask why $h_k(x) \to 0$ as $x \to a$.

I think the easiest way to see this is to use l'Hopital's rule. So consider $$ \frac{f(x) - (f(a) + f'(a) (x-a) + \cdots + f^k(a) (x-a)^k / k!)}{(x-a)^k} = h_k(x).$$ Let us evaluate the left hand side as $x \to a$, and in so doing evaluate the limit $\lim_{x \to a} h_k(x)$.

Clearly the left hand side appears like $0/0$, and differentiating once gives another limit in the indeterminate form $0/0$. But after applying l'Hopital's rule $k$ times, the left hand side becomes $$ \lim_{x \to a} \frac{f^k(x) - f^k(a)}{k!} = 0.$$ This answers your first question.

Your second question is why $h_k(x) \to 0$ as $k \to \infty$. And the short answer is that this is not always true, even if $f$ is infinitely differentiable. Sometimes it's true, but sometimes it is not. Functions for which this is true for every $a$ are called analytic and are very special --- in many ways, they are the nicest functions.

---

As an aside, I would point out that I wrote a note for my students some years ago in an attempt to give a somewhat better look at what Taylor series and polynomials really are. In it, I discuss the concept of the error terms in ways that some of my students have found very helpful.

Answer 2

Do you understand that this is not always true? There exist infinitely differentiable function such that the Taylor series exist and converges for all x but not to the original function!

$f(x)= e^{-\frac{1}{x^2}}$ if x is not 0, f(0)= 0, is such a function. It is not difficult to show that every derivative is $e^{-\frac{1}{x^2}}$ times a polynomial in x, so every derivative has value 0 at x= 0. That is, the Taylor's series, evaluated at 0, is identically 0 but the function is clearly not always 0.

Answer 3

Here, we present a straightforward approach to proving that the Peano form of the remainder, $h_k(x)$, approaches zero as $x\to a$. To that end, we proceed.

Let $f(x)$ be $k$ times continuously differentiable and let $h_k(x)$ be defined as

$$\bbox[5px,border:2px solid #C0A000]{h_k(x)=\frac{f(x)-\sum_{j=0}^k \frac{f^{(j)(a)}}{j!}(x-a)^j}{x^k}}$$

Then, applying L'Hospital's Rule successively $k$ times reveals

$$\begin{align} \lim_{x\to a}h_k(x)&=\lim_{x\to a}\frac{f(x)-\sum_{j=0}^k \frac{f^{(j)(a)}}{j!}(x-a)^j}{x^k}\\\\ &=\lim_{x\to a}\frac{f'(x)-\sum_{j=1}^k \frac{f^{(j)(a)}}{(j-1)!}(x-a)^{j-1}}{x^k}\\\\ \vdots\\\\ &=\lim_{x\to a}(f^{k}(x)-f^{k}(a))\\\\ &=0 \end{align}$$

Therefore, we find that

$$\bbox[5px,border:2px solid #C0A000]{f(x)=\sum_{j=0}^k \frac{f^{j}(a)}{j!}(x-a)^j+h_k(x)(x-a)^k}$$

where $\lim_{x\to a}h_k(x)=0$.