Something that has bugged me since university. Why does the Taylor Series have that specific form? For example there is a division by n! - why not (say) (n^2)!
How does one get to the Taylor Series? It does not strike me as "intuitively obvious" that it
is a representation of a function as an infinite sum of terms that are calculated from the values of the function's derivatives at a single point.
Was this all trial and error or am I missing something here?
On
The idea came from trying to approximate a function as a polynomial. If you can only measure the function at one point, say $a\in \mathbb{R}$, the best you can say is that $f(x) \approx f(a)$ but you want more precision.
If you know the first derivative (defined as the slope of the tangent line at $a$), then you can approximate your function as a line with the equation $$y - f(a) = f'(a)(x - a),$$ which is equivalent to $$ f(x) \approx f(a) + f'(a)(x - a). $$ One can continue the analogy to add higher order terms, and you get $f$ with various levels of approximation: $$ \begin{split} f(x) &\approx f(a) \\ &\approx f(a) + f'(a)(x-a) \\ &\approx f(a) + f'(a)(x-a) + \frac{f''(a)}{2} (x-a)^2 \\ &\approx f(a) + f'(a)(x-a) + \frac{f''(a)}{2} (x-a)^2 + \frac{f'''(a)}{3!} (x-a)^3\\ &\ldots \end{split} $$ Each successive level adds more complexity to the approximation, and decreases the error estimate, at the cost of requiring more information.
An amazing result due to Taylor, is that under some nice conditions on $f$, the right-hand side actually converges to the left-hand side when an infinite sum is taken, i.e. $$ f(x) = \sum_{n=0}^\infty \frac{f^{(n)}(a)}{n!} (x-a)^n. $$
How do you generally come up with the correct coefficients? If you think of a generating function for $f(x)$, writing $$ f(x) = \sum_{n=0}^\infty b_n (x-a)^n $$ it is easy to see that $b_0 = f(a)$ and $b_1 = f'(a)$, etc.
On
The factorials happen when you find higher derivatives of polynomials.
For example the second derivative of $x^2$ is $2!$
The fifth derivative of $x^5$ is $5!$ and so forth.
Finding the coefficients of the Taylor polynomials requires finding higher derivatives of your function at the center and making it equal to the higher derivatives of polynomial at the center.
The division by factorials are the result of identifying those derivatives.
On
Quiz:
Consider the polynomial
$$P(x):=ax^3+bx^2+cx+d.$$ You are requested to guess the coefficients. All you are allowed to do is to query values of the polynomial and its derivatives at $x=0$.
Solution:
Obviously, $P(0)=d$.
Then, observing
$$P'(x)=3ax^2+2bx+c,$$ we have
$$P'(0)=c.$$
Next, following this principle,
$$P''(0)=2\cdot b$$ and
$$P'''(0)=3\cdot2\cdot a.$$
Hence,
$$P(x)=P(0)+P'(0)x+P''(0)\frac{x^2}2+P'''(0)\frac{x^3}{3!}.$$
Now generalize to a polynomial of arbitrary degree.
On
Well since I can't immediately find a duplicate, here's one of my favourite approaches to proving Taylor's theorem with the remainder.
As an example, we will derive the Taylor expansion to $x^3$ with an $x^4$ remainder term. Assume the $4$th derivative of a function $f$ is bounded over $x\in[0,1]$:
$$M\le f''''(x)\le N\qquad\forall x\in[0,1]$$
Integrate both sides as follows:
$$\int_0^xM~\mathrm dt\le\int_0^xf''''(t)~\mathrm dt\le\int_0^xN~\mathrm dx\qquad\forall x\in[0,1]$$
Each integral is easy to evaluate and gives us
$$Mx\le f'''(x)-f'''(0)\le Nx\qquad\forall x\in[0,1]$$
Integrating again gives us
$$\int_0^xMt~\mathrm dt\le\int_0^xf'''(t)-f'''(0)~\mathrm dt\le\int_0^xNt~\mathrm dt\qquad\forall x\in[0,1]$$
which reduces down to
$$\frac12Mx^2\le f''(x)-f''(0)-f'''(0)x\le\frac12Nx^2\qquad\forall x\in[0,1]$$
Integrating again gives us
$$\frac1{2\cdot3}Mx^3\le f'(x)-f'(0)-f''(0)x-\frac12f'''(0)x^2\le\frac1{2\cdot3}Nx^2\qquad\forall x\in[0,1]$$
and finally, after one more iteration,
$$\frac1{2\cdot3\cdot4}Mx^4\le f(x)-f(0)-f'(0)x-\frac12f''(0)x^2-\frac1{2\cdot3}f'''(0)x^3\le\frac1{2\cdot3\cdot4}Nx^3\qquad\forall x\in[0,1]$$
Note that with each integration, we introduce another factor in the denominators, leading to a factorial. The above can be rewritten as
$$f(x)=f(0)+f'(0)x+\frac12f''(x)+\frac16f''(0)x^3+\epsilon(x)\qquad\forall x\in[0,1]$$
$$\frac1{24}Mx^4\le\epsilon(x)\le\frac1{24}Nx^4$$
Starting with the assumption that the $(n+1)$th derivative of $f$ is bounded will allow you to develop the Taylor expansion to the $x^n$ term, with a $\mathcal O(x^{n+1})$ remainder.
The basic principle is that you want to match the function's value, first derivative, second derivative, third derivative, etc. at a single point. Such a series is not guaranteed to actually equal the function at all points, but it will for an important class of functions called analytic functions.
This point is often chosen to be $x = 0$. One reason is convenience: At $x = 0$, the value is determined solely by the constant coefficient, the first derivative solely by the linear coefficient, the second derivative solely by the quadratic coefficient, the third derivative solely by the cubic coefficient, and so forth.
What's more, we can find out what these coefficients are by taking those derivatives. For instance, suppose we denote the third coefficient by $a_3$. Then the third derivative of $a_3x^3$ is
$$ \frac{d^3}{dx^3}a_3x^3 = \frac{d^2}{dx^2}3a_3x^2 = \frac{d}{dx}6a_3x = 6a_3 $$
Setting this equal to $f'''(0)$ gives
$$ a_3 = \frac{f'''(0)}{6} $$
This generalizes in the expected way for all coefficients.