Using Taylor series, establish the error term for the formula $f′′(x)≃ \frac{f(x+h)−2f(x)+f(x−h)}{h^2}$. Assume f(x) and all its derivatives have similar magnitudes, determine the value of $h=1/(2^m)$ that gives the most accurate approximation of f''(x).
I understand that by using Taylor series, the error term is O( $\frac{h^4f''''(x)}{24}$ ). But how to get h?
The Taylor polynomial of degree $3$ with Lagrange form for remainder reads $$\begin{align}f(x-h)&=f(x)-hf^{\prime}(x)+\frac12h^2f^{\prime\prime}(x)-\frac16h^3f^{\prime\prime\prime}(x)+\frac1{24}h^4f^{(4)}(\xi_1)\\ f(x+h)&=f(x)+hf^{\prime}(x)+\frac12h^2f^{\prime\prime}(x)+\frac16h^3f^{\prime\prime\prime}(x)+\frac1{24}h^4f^{(4)}(\xi_2)\end{align}$$ With $x-h<\xi_1<x<\xi_2<x+h$. Adding, then subtracting $2f(x)$ we have $$\begin{align}f(x-h)-2f(x)+f(x+h)&=h^2f^{\prime\prime}(x)+\frac{h^4}{24}\left(f^{(4)}(\xi_1)+f^{(4)}(\xi_2)\right)\\ &=h^2f^{\prime\prime}(x)+\frac{h^4}{12}f^{(4)}(\xi)\end{align}$$ Where the intermediate value theorem has assured us that there must be a $\xi$ such that $x-h<\xi_1<\xi<\xi_2<x+h$ and $f^{(4)}(\xi)=\frac12\left(f^{(4)}(\xi_1)+f^{(4)}(\xi_2)\right)$, assuming a continuous $f^{(4)}(x)$. That means that $$\frac{f(x-h)-2f(x)+f(x+h)}{h^2}=f^{\prime\prime}(x)+\frac{h^2}{12}f^{(4)}(\xi)$$ Assuming IEEE-754 single precision arithmetic with $24$ bits of precision, the error in $f(x-h)-2f(x)+f(x+h)=h^2f^{\prime\prime}(x)$ is about $2^{-24}f(x)$ in magnitude so the roundoff error in $f^{\prime\prime}(x)$ is about $\frac{2^{-24}}{h^2}f(x)$. The truncation error is about $\frac{h^2}{12}f(x)$ and these $2$ errors have the same magnitude when $h^4=12\times2^{-24}$; $h\approx0.029\approx2^{-5}$. For double precision arithmetic with $53$ bits of precision, $h^4=12\times2^{-53}$; $h\approx1.9\times10^{-4}\approx2^{-12}$.