How is Taylor expansion a generalization of linear approximation?

Question

The concept of derivative is related to linear approximation of a function: $$f(x)\approx f(a)+f'(a)(x-a)$$

I was told that this linear approximation is generalized by the Taylor series. What does this mean?

Answer 1

Suppose $f(x)$ has derivatives of all orders at $x=a$ and let $T_n(x)$ denote the $n$th degree polynomial approximation of $f$ at $x=a$.

Linear approximation:

$T_1(x)$ matches $f(x)$ in function value and first derivative at $x=a$. That is, $$ T_1(x)=c_1(x-a)+c_0,\ T_1(a)=f(a),\ T_1'(a)=f'(a)\implies c_1=f'(a),\ c_0=f(a). $$

Quadratic approximation:

$T_2(x)$ matches $f(x)$ in function value, first, and second derivative at $x=a$. That is, \begin{align} T_2(x)&=c_2(x-a)^2+c_1(x-a)+c_0,\\ T_2(a)&=f(a),\ T_2'(a)=f'(a),\ T_2''(a)=f''(a) \end{align} implies $$c_2={f''(a)\over 2},\ c_1=f'(a),\ c_0=f(a). $$

Cubic approximation:

$T_3(x)$ matches $f(x)$ in function value, first, and second derivative at $x=a$. That is, \begin{align} T_3(x)&=c_3(x-a)^3+c_2(x-a)^2+c_1(x-a)+c_0,\\ T_3(a)&=f(a),\ T_3'(a)=f'(a),\ T_3''(a)=f''(a),\ T_3'''(a)=f'''(a)\\ \end{align} implies $$c_3={f'''(a)\over 3!},\ c_2={f''(a)\over 2},\ c_1=f'(a),\ c_0=f(a). $$

$n$th order approximation:

$T_n(x)$ matches $f(x)$ in $0,1,2,\dots,n$th order derivatives at $x=a$. That is, \begin{align} T_n(x)&=\sum_{k=0}^n c_k(x-a)^k\\ T^{(k)}_n(a)&=f^{(k)}(a),\ k=0,1,\dots,n\\ \end{align} implies $$c_k={f^{(k)}(a)\over k!},\ k=0,1,\dots,n. $$

Taylor series:

The Taylor series for $f(x)$ at $x=a$ is $$\sum_{k=0}^\infty {f^{(k)}(a)\over k!}(x-a)^k,$$ is just $\displaystyle\lim_{k\to\infty}T_k(x)$.

Answer 2

With a linear equation you can match the value and slope (first derivative) of a differentiable function at some point.

With a quadratic you can match the second derivative too (roughly curvature), with a cubic you can match the third derivative, and so on ...

If your function is sufficiently well-behaved, and you line up the best fit polynomials of each degree you see that they extend one another, so you can create an infinite series which matches every derivative - and under the right conditions the function expressed by the series is the same as the original function.

The various polynomials can be used to estimate the value of the function.