Quadratic/ Cubic/ etc approximations without the Taylor series

Question

It's easy to convince someone that the linear approximation is the best line which approximates a function at a point because everyone learns early that the derivative of a function is just the slope of that function at the point.

However...

Assuming I'd never heard of Taylor series, is there some way you could convince me that $f(a)-f'(a)(x-a) - \frac 12f''(a)(x-a)^2$ is the best quadratic function that describes the function $f$ at the point $x=a$?

Bonus points: How about the best cubic or quartic or quintic or .... approximation?

Answer 1

Write $f(x)\approx a_0+a_1(x-a)+a_1(x-a)^2$. Then, determine coefficients by taking successive derivatives and letting $x=a$.

So, $f(a)=a_0$. $f'(a)=a_1$. $f''(a)=2a_2$.

The same procedure can be used for any order of "approximation." However, this does not establish the validity of the process. Nor does it provide any estimate of the size of the error. It only gives a heuristic way of looking at the elegant development that is Taylor's expansion.

Answer 2

Well, to be awkward and annoying, you can define the derivative as the number $f'(a)$ so that $$ \frac{f(a+h)-f(a)-h f'(a)}{h} \to 0 $$ as $h \to 0$, i.e. so that $$ f(a+h) = f(a) + h f'(a) + o(h). $$ In a similar way, you can define the second derivative as the number $f''(a)$ such that $$ \frac{f(a+h)-f(a)-h f'(a)-h^2 f''(a)/2 }{h^2} \to 0, $$ and so on. That there is only one such number $f^{(k)}(a)$ for each $k$ up to $n$ is exactly what it means for a function to be $n$-times differentiable.