Finite difference error for twice differentiable function

Question

Assume that $f\in C^2(\mathbb{R})$. Then I want to approximate $f''$ by some finite difference as \begin{align} f''(x)\approx\frac{f(x+h)-2f(x)+f(x-h)}{h^2} \end{align} But since $f$ has only two derivatives, a Taylor expansion shows that \begin{align} f(x+h)=f(x)+f'(x)h+\frac{1}{2}f''(x+\Theta h)h^2,\qquad \Theta\in (0,1). \end{align} Hence \begin{align} \frac{f(x+h)-2f(x)+f(x-h)}{h^2}=\frac{1}{2}(f''(x+\Theta h)+f''(x-\tilde{\Theta}h)) \end{align} So is there a way to get some error like \begin{align} f''(x)=\frac{f(x+h)-2f(x)+f(x-h)}{h^2}+ O(h)? \end{align} which would be easy by Taylor's theorem if $f\in C^3(\mathbb{R})$ but I cannot assume this here. I only see because of continuity of $f''$ that \begin{align} f''(x+\Theta h)=f''(x) + o(1),\quad h\downarrow 0. \end{align}

Answer 1

So Taylor's theorem says the following: For $f \in C^2(\mathbb{R})$ you can control $$f(x \pm h) = f(x) \pm f'(x) h + \frac{1}{2}f''(x)h^2 + p(x)h^2$$ which is the Peano form of the remainder. For this particular formulation, your error $$E := f''(x) - \frac{f(x+h)-2f(x)+f(x-h)}{h^2} $$ is of order $1$.

You could take a look at other forms of the remainder. Mean-value formulas of the remainder based on $f'''$ are by assumption clearly invalid. However, if $f''$ is absolutely continuous and $f'''$ is integrable (but not continuously differentiable) you could use a integral form of the remainder. However, this is also of Order $h^2$, thus your overall error will again be or order $1$.