Taylor polynomial approximation

Question

How do you determine if adding more terms to the Taylor polynomial will improve its approximation of $f(p)$ or in other words, how do you determine if a Taylor series converges for a particular value of $x$? I understand the concept but not conceptually "enough." A clear explanation to help clear the road would be appreciated.

Answer 1

This is the subject of the remainder term. Statements of Taylor's theorem generally come with one or more formulas for the remainder in some fashion generalizing the mean value theorem that is often very amenable to bounding.

For example, using the Lagrange form of the remainder for the Taylor series to $e^x$, we have

$$ e^x = \left( \sum_{i=0}^k \frac{x^i}{i!} \right) + e^c \frac{x^{k+1}}{(k+1)!} $$

for some $c$ between $0$ and $x$. That is, if $x \geq 0$ then $0 \leq c \leq x$ and if $x \leq 0$ then $x \leq c \leq 0$.

So if you can divine any upper bound at all on the value of $e^c$, you can use this form to get bounds on the error of the $k$-th order approximation to $e^x$ is. For example, if $x < 0$, we automatically know $0 < e^c \leq 1$. Or if $0 < x < 1$, we know $1 \leq e^c < 4$.

---

e.g. if I was in the $0 < x < 1$ case and I wanted to know what $k$ I needed to get the error on $e^x$ to within $0.01$, I would first do the above analyis to get

$$ \left( \sum_{i=0}^k \frac{x^i}{i!} \right) + 1 \cdot \frac{x^{k+1}}{(k+1)!} \leq e^x \leq \left( \sum_{i=0}^k \frac{x^i}{i!} \right) + 4 \cdot \frac{x^{k+1}}{(k+1)!} $$

To make things easier, I'll throw in the bounds on $x$ as well:

$$ \left( \sum_{i=0}^k \frac{x^k}{k!} \right) < e^x < \left( \sum_{i=0}^k \frac{x^k}{k!} \right) + \frac{4}{(k+1)!} $$

where on the left side I've used $0 < x$ and on the right side I've used $x < 1$.

From this, I can see that the error is less than $4 / (k+1)!$. If I can make this value less than $0.01$, that ensures the error is even smaller, and I'm happy.

Sometimes this leaves you with a problem you'll have to spend some effort to solve. However, this problem is too easy to bother with complicated methods: I'll just use brute force:

• $k=0$ says the error is less than $4$
• $k=1$ says the error is less than $2$
• $k=2$ says the error is less than $0.7$
• $k=3$ says the error is less than $0.17$
• $k=4$ says the error is less than $0.04$
• $k=5$ says the error is less than $0.007$

so I'll use $k=5$, and I'm guaranteed that

$$ \left( \sum_{i=0}^5 \frac{x^i}{i!} \right) < e^x < \left( \sum_{i=0}^5 \frac{x^i}{i!} \right) + 0.01 $$

Note that I'm pretty sure that $k=4$ is actually good enough as well. But it would have taken more effort to prove this fact. Using $k=5$ doesn't mean we get wrong results -- it just means we spent a little more effort getting a good approximation than we really needed to. But quite possibly, the effort to find $k=4$ works would be a lot greater, so we're actually better off simply using the $k=5$ from the simple analysis.

---

In this case, one could even make a clever argument in the $x>0$ case for this particular function: clearly we have

$$ \left( \sum_{i=0}^k \frac{x^i}{i!} \right) \leq e^x $$

and we also have $e^c \leq e^x$. So we also know

$$ e^x \leq \left( \sum_{i=0}^k \frac{x^i}{i!} \right) + e^x \frac{x^{k+1}}{(k+1)!} $$ $$ e^x \left(1 - \frac{x^{k+1}}{(k+1)!} \right) \leq \left( \sum_{i=0}^k \frac{x^i}{i!} \right) $$

If you choose $k$ large enough so that $x^{k+1} < (k+1)!$, then we have

$$ e^x \leq \left( \sum_{i=0}^k \frac{x^i}{i!} \right) \left(1 - \frac{x^{k+1}}{(k+1)!} \right)^{-1} $$

$e^x$ grows really fast, so a bound like this that is multiplicative rather than additive may well be good enough for your purposes. A great deal of the time when using the remainder theorem you don't have to be so creative, though: there is often a straightforward way to come up with a bound on the error term. And it's usually okay if it's a really sloppy bound -- that just means you might wind up using a few more terms than you actually needed to, but that just means you'll be getting even better approximations than you were searching for.

---

Incidentally, if you don't actually care about bounds and just want to see convergence as $k \to \infty$, then the above can still apply; it's just that you can afford to be very sloppy in your bounds, so long as they still converge. e.g. for $x>0$, above I've proven that

$$ \left( \sum_{i=0}^k \frac{x^i}{i!} \right) \leq e^x \leq \left( \sum_{i=0}^k \frac{x^i}{i!} \right) \left(1 - \frac{x^{k+1}}{(k+1)!} \right)^{-1} $$

but to see convergence as $k \to \infty$, it is good enough, and probably even easier, to just use

$$ \left( \sum_{i=0}^k \frac{x^i}{i!} \right) \leq e^x \leq \left( \sum_{i=0}^k \frac{x^i}{i!} \right) + e^x \frac{x^{k+1}}{(k+1)!} $$

The ratio test proves that the infinite sum converges as $k \to \infty$. Squeeze theorem applied to either of the above inequalities shows that the infinite sum must converge to the value of $e^x$. And therefore, we can truly say that

$$ e^x = \sum_{i=0}^\infty \frac{x^i}{i!} $$

Functions $f(x)$ that have the property that their Taylor series actually converges to $f(x)$ are called analytic functions.

It may be interesting to know that there are functions whose Taylor series converges, but to something different than the function! So asking about convergence is an important question when using the infinite sums rather than the finite approximations with remainder.

Once you realize how easily a function could fail to be analytic, it a bit of a surprise to realize how many of the functions we're used to using really do turn out to be analytic.