I am reading one-variable calculus book where it explains Taylor series and little confused with the following terms:
(1) Expand $f(x)$ at $x_0$
(2) Evaluate $f(x)$ at x
(3) Best Affine, linear, quadratic, cubic, and other approximations
My understanding is roughly...:
(1) Express an arbitrary function $f(x)$ in terms of polynomials with various terms of coefficients being the derivative values all evaluated at the point of expansion $x_0$.
(2) Given you have Taylor series from (1), the RHS is basically $n^{th}$degree polynomial with argument $x$. So just plug in a number in $x$ if you want to "evaluate" $f$.
(3) There is a class of functions you can use Taylor's formula to get the $n^{th}$ degree polynomial which is exactly $f(x)$. But this may not be the case, and in that case, we have Taylor's formula with remainder indicating the polynomial form is only approximation. The best affine approximation is of the form $f(x_o)+f'(x_o)(x-x_o)$ where as the linear approximation is $f'(x_o)$. Notch up one to $n=2$, you have power term so it is quadratic, and so forth.
My questions:
(1) I understand the goal is turning an arbitrarily function to a polynomial expression, but I don't quite get how this "expand" and "evaluate" machinery works as big picture.
(2) I remember in high school, teacher would say "we can ignore the remainder term in Taylor's Theorem, because it is so small". To what class of functions does this apply, and when is an exception to this rule?
(3) In introductory calculus book, it just mentions polynomials are easier to work with and have nice properties, but in introductory analysis course, I know it goes beyond this with power series, uniform convergence, convolutions, Borel Theorem, and the Weierstrass approximation. Can someone provide a big picture from the beginning to Weierstrass? I simply want to look at the whole theme of "approximation" and in the context of optimization. The main motivation from the univariate calculus book in covering Taylor is to develop relative extrema test. So I am reading the section in the context of optimization.
Thank you.
I'll try to address some of the misunderstandings I think you have based on the question and the comments. Someone else can tackle the problem given in $(3)$ of explaining everything about Taylor polynomials and Taylor series from the ground up if they want (though I doubt anyone will).
First off, it sounds like you're conflating Taylor polynomials with Taylor series. A Taylor polynomial is just what it sounds like -- a polynomial. How can a polynomial "not converge"? Polynomials are defined everywhere.
Taylor's theorem says that a function which is $k$-times differentiable at a point $x$ can be approximated in some neighborhood of the point $x$ by the $k$th degree Taylor polynomial of the function centered at $x$. And depending on how you prove the theorem, you have different ways of bounding the error in your approximation. You're right the Taylor polynomials are only good approximations in some neighborhood of the point $x$ -- it depends on the function and your tolerance in the error as to how big that neighborhood is.
One can approximate a function by a Taylor series if it is infinitely differentiable at the point $x$. To do this one considers the sequence of degree $k$ Taylor polynomials as $k\to \infty$. But even if that sequence converges that doesn't mean that the function necessarily equals its Taylor series even in some small neighborhood of the point $x$. Sometimes the remainder terms don't tend to $0$ even in the limit. A function which does equal its Taylor series in some neighborhood of every point is called an analytic function.
The terms "best ____ approximation" mean the function of type ____ whose remainder term vanishes fastest near the point $x$. For example, the best affine approximation of a function $f$ near $x_0$ is the function $L(x) = f(x_0) + f'(x_0)(x-x_0)$. The reason this is the best is that of all of the affine functions $T(x) = A+Bx$, the above is the only one such that $\lim_{x\to x_0} \frac{f(x)-T(x)}{x-x_0} = \lim_{x\to x_0} \frac{\text{remainder}}{x-x_0}=0$. It turns out that the Taylor polynomials are always the best polynomial approximations of a function in some neighborhood of $x$.
One other note: continuous differentiability does not mean infinitely differentiable. If a function is continously differentiable at a point that means that it is differentiable and its derivative is continuous. But if a function is twice differentiable -- without even having a continuous second derivative -- then it is automatically continuously differentiable because differentiability implies continuity. So if it's twice differentiable then that means that its derivative is differentiable which means that the derivative is continuous. So don't mix up those ideas. Any $k$-times differentiable function is at least $(k-1)$-times continuously differentiable.