Why is the error of $O(h^2)$ when using Taylor expansion and centered approximation for the first derivative

Question

We know that the approximation of the first derivative by centered approximation is given by $$ f'(x) = \frac{f(t+h) - f(x-h)}{2h} + O(h^2)$$ The quality of the above approximation is determined by the error. For smaller values of $h>0$ the error should be as small as possible. To determine this error we use the Taylor expansion

\begin{align*} y(t+h) &= y(t) + y'(t)h + \frac{y''(t)}{2}h^2+\frac{y'''(t)}{3!}h^3+ O(h^4)\\ y(t-h) &= y(t) - y'(t)h + \frac{y''(t)}{2}h^2-\frac{y'''(t)}{3!}h^3+ O(h^4)\\ y(t+h) - y(t-h) &= 2y'(t)h + 2\frac{y'''(t)}{3!}h^3 + O(h^4)\\ \frac{y(t+h)-y(t-h)}{2h}-y'(t) &= \frac{y'''(t)}{3!}h^2 + O(h^3) = O(h^2) \end{align*}

an thus we conclude $$ y'(t) = \frac{y(t+h)-y(t-h)}{2h}+O(h^2) $$

Here is my question

We could have arbitrarily decided to take the Taylor expansion with the polynomial of fourth and fifth order namely writing the first line as follows: $$ y(t+h) = y(t) + y'(t)h + \frac{y''(t)}{2}h^2+\frac{y'''(t)}{3!}h^3+\frac{y''''(t)}{4!}h^4+\frac{y'''''(t)}{5!}h^5+ O(h^6) $$ Thus the last line would have been: $$ \frac{y(t+h)-y(t-h)}{2h}-y'(t) = \frac{y'''(t)}{3!}h^2 + \frac{y'''''(t)}{5!}h^4 + O(h^5) = O(h^4) $$ and conclude that the error is of order $= O(h^4)$

Why is this not true?

Answer 1

Because for smaller value of $h$ which is less than $1$, $h^2$ is greater than $h^5$. Hence, if the error of an algorithm is $O(h^2)$ is in $O(h^5)$ for small values of $h$ and not vice versa.

Answer 2

The correct last line is

$$ \frac{y(t+h)-y(t-h)}{2h}-y'(t) = \frac{y'''(t)}{3!}h^2 + \frac{y'''''(t)}{5!}h^4 + o(h^4) = \Theta(h^2) $$ (unless $y'''(t)=0$).