Most accurate finite difference result for first derivative

Question

I got a problem in my assignment: obtain the most accurate finite difference results possible for the first derivative of f (x) = exp(cos(x)) at x=1, h = 0.5, 0.25, 0.125,...2^{16}. I have to do this in MATLAB. Can any one explain how to find such accurate result, so that I can implement it in MATLAB

Answer 1

See here and here. You can get arbitrarily accurate results by using arbitrarily high order approximations, so your question is somewhat underspecified.

Answer 2

Using Taylor Expansion, there are many finite difference derivative estimates one can formulate.

If you have your data in hand, and will not solve a system of DE, then my choice would be a 1 step central difference which has convergence order $h^2$. Other possible "easy" approaches can be forward difference or backward difference.

Answer 3

You can compute the exact derivative to compare against.

The central difference formula $$ f'(x)=\frac{f(x+h)-f(x-h)}{2h}+O(h^2) $$ has two error contributions, the discretization error $\sim f''(x) h^2$ and the evaluation error $\sim |f(x)|\mu/h$ where $\mu$ is the floating point machine constant.

The overall error is minimal where both contributions are about equal, which happens at $h\sim\sqrt[3]\mu\approx 2^{-17}\approx 8·10^{-6}$, the last for double precision floats, with an error of about $2^{-34}\approx 6.4·10^{-11}$

---

If you use then Richardson extrapolation or other methods to get a higher order formula, the next best result would be $O(h^4)$. By the same estimates, the best result is obtained for $h\sim\sqrt[5´]\mu\approx 2^{-10}\approx 10^{-3}$ with error in the region of $2^{-40}\approx 10^{-12}$.

---

This trend continues, the higher the order, the larger the intersection between the error contributions. However, the gains in using ever higher order formulas quickly reduce to a trickle.