Finite difference second derivative: Error analysis

Question

I want to approximate $f''(x)$ using finite differences. From \begin{align*} f(x+h)=f(x)+f'(x)h+\frac{1}{2}f''(x)h^2+\frac{1}{6}f'''(x)h^3+O(h^4),\\ f(x-h)=f(x)-f'(x)h+\frac{1}{2}f''(x)h^2-\frac{1}{6}f'''(x)h^3+O(h^4), \end{align*} we easily obtain \begin{align} f''(x)=\frac{f(x+h)-2f(x)+f(x-h)}{h^2}+O(h^2). \end{align} Alternatively, we can apply centered finite differences twice with step size $h/2$ and error $O(h^2)$ to write: \begin{align} f''(x) &= \frac{f'(x+h/2)-f'(x-h/2)}{h}+O(h^2)\\ &=\frac{\frac{f(x+h)-f(x)}{h}-\frac{f(x)-f(x-h)}{h}+O(h^2)}{h}+O(h^2)\\ &=\frac{f(x+h)-2f(x)+f(x-h)}{h^2}+O(h)+O(h^2). \end{align} I do not understand the inconsistency between the two approaches. Should the $O(h)$ error term above vanish?

Answer 1

The leading error terms for $f'\left(x+\frac{h}{2}\right)$ and $f'\left(x-\frac{h}{2}\right)$ are $-\frac{h^2}{24}f'''\left(x+\frac{h}{2}\right)$ and $\frac{h^2}{24}f'''\left(x-\frac{h}{2}\right),$ respectively. You can see this, for example, for $f'\left(x+\frac{h}{2}\right)$ by subtracting the Taylor series expansions of $f(x+h)$ and $f(x)$ about $x+(h/2)$ and then dividing the result by $h.$ So, we get

$$\dfrac{f'\left(x+\frac{h}{2}\right)-f'\left(x-\frac{h}{2}\right)}{h} = \dfrac{f(x+h)-2f(x)+f(x-h)}{h^2}+\dfrac{h}{24}\left[f'''\left(x-\frac{h}{2}\right)-f'''\left(x+\frac{h}{2}\right)\right]+\cdots.$$

As you pointed out, it appears like the leading error term is $O(h),$ but if you Taylor expand $f'''\left(x-\frac{h}{2}\right)$ and $f'''\left(x+\frac{h}{2}\right)$ about $x,$ you will see that the term inside the square bracket above simplifies to $-hf''''(x)+\cdots,$ and therefore the leading error term is indeed $O(h^2)$ as expected.

Answer 2

You get $$f''(x)=\frac{\frac{f(x+h)-f(x)}{h}-\frac{f(x)-f(x-h)}{h}+O(h^2)}{h}$$by reasoning that the two approximations to $f'$ both have leading error $O(h^2)$ and therefore their difference will have error $O(h^2)$.

In fact they have the same leading error term which will cancel upon subtraction, and leave you with the result that you were expecting.