Assume that $n+1$ different grid points $x_0,...,x_n$ are given.
Exercise: determine the coefficients $a_i$ such that de order $(k)$ of the error: $$\left|\,f'(x) - Q(h)\right| = O(h^k)$$ is maximal. We have that $Q(h)$ is given by: $$Q(h) = \sum_{i = 0}^n a_i f_i, \text{ where }f_i = f(x_i).$$
Approach: develop every $f_i$ in a Taylor series of high enough order. Determine $a_0, ..., a_n$ such that the order of the error is maximal. Using the central difference method with the points $x_{-1} = x - h,\,\,x_0 = x$ and $x_1 = x + h$, we get that $a_{-1} = \frac{-1}{2h},\, a_0 = 0$ and $a_1 = \frac{1}{2h}$. We now have: $$Q(h) = \dfrac{f(x+h) - f(x-h)}{2h}.$$
Questions:
- Apparently, with this particular $Q$ we have that the error $\left|\,f'(x) - Q(h)\right| = O(h^k)$ is of order $O(h^2)$. I don't understand why; shouldn't it be of $O(h)$? Since $h$ is the term with the highest growth rate? I found this example on Wikipedia.
- Why would you want the order $(k)$ of the error to be maximal? I assume you want the error itself to be a small as possible right? Wouldn't this be the case when the order of the error is a small as possible?
Thanks in advance!
Assuming $f $ sufficiently smooth and using Taylor series as $h\to 0$, $$\frac{f(x+h) - f(x-h)}{2h} = f'(x) + \frac{1}{6} h^2 f^{(3)}(x) + \dots$$ Here, there is no term in $h$, but there is one term in $h^2$. Thus, the error is $O(h^2)$.
The fact is that in the vicinity of $h=0$, the error has a smaller absolute value if the order is bigger. Indeed, if $0<h<1$, then $0<h^{p+1}<h^p$. This can be observed graphically too: