Given a central difference formula:
$$f'(x)\approx D(x)=\frac{f(x+h)-f(x-h)}{2h}$$
How can I find the closest error estimation $R(x)$ such that:
$$|D(x)-f'(x)|\le R(x)$$
I'm not much experienced in mathematics, so I prefer a simple and clear explanation.
If you write a Taylor expansion of $f(x + h)$ around $x$,
$$ f(x + h) = f(x) + hf'(x) + \frac{h^2}{2}f''(x) + \frac{h^3}{3!}f'''(x) + \cdots \tag{1} $$
Replacing $h\to -h$ in (1)
$$ f(x - h) = f(x) - hf'(x) + \frac{h^2}{2}f''(x) - \frac{h^3}{3!}f'''(x) + \cdots \tag{2} $$
Now, subtracting (1) from (2):
$$ f(x + h) - f(x - h) = 2f'(x)h + 2\frac{h^3}{3!}f'''(x) + \cdots \tag{3} $$
Rearranging
$$ f'(x) = \frac{f(x+h) - f(x-h)}{2h} - \frac{h^2}{3}f'''(x) + \cdots \tag{4} $$
Or
$$ \left| f'(x) - \frac{f(x+h) - f(x-h)}{2h}\right| = \left|\frac{h^2}{3}f'''(x) + \cdots \right| $$