If Forward Euler is an over-approximation does that imply that backward Euler is an under-approximation?

Question

I noticed that in many cases I've tried using forwards and backwards Euler, the solution is between the two. I'm curious if one estimate over estimates when the other underestimates. This is how I tried to look at the problem. I tried to simplify things by looking at the case $$\frac{dx}{dt}=f(x),\ x(a)=x_0,$$ and only consider one time-step when we want to predict $x(a+\Delta t)$. This problem is equivalent to problem where $$\frac{dy}{dt}=g(y),\ y(0)=0$$ and we want to find $y(\Delta t)$ where $y=x-x_0$ and $g(y)=f(y+x_0)$ by the time invariance principle. Explicit Euler predicts that $$y(\Delta t)=g(y(0))\Delta t=g(0)\Delta t$$ and Implicit Euler predicts that $$y(\Delta t)=g(y(\Delta t))\Delta t.$$ We can assume $\Delta t=1$ by replacing $g(y)$ with $$h(y)=\frac{1}{\Delta t}g(y)$$ So Explicit Euler predicts that $$y(1)=h(0),$$ Implicit Euler predicts that $$y(1)=h(y(1)),$$ and the correct value of $y(1)$ is $$y(1)=\int_0^1h(x(t))dt.$$ So basically I want to compare $h(0)$, the fixed point of $h$, and $$\int_0^1 h(x(t))dt.$$ I'm wondering if there is an example of a function $h$ such that both $h(0)$ and the fixed point of $h$ are strictly greater than $$\int_0^1h(x(t))dt$$ because that would prove that the exact solution is not between the two versions of Euler. Otherwise I would like to know if it is never possible in which case I'm wondering if my argument could be extended to multiple iterations (what makes it harder is that the starting point of the exact solution is not the same as the starting point of the the numerical methods by the second iteration)

Answer 1

Yes, the errors of both methods are somewhat complimentary, if you take the average the next order error cancels out and you get Heun's method or the implicit trapezoidal method, both of second order.

In formulas, the Taylor expansion of a step in the exact solution is (obvious arguments are left out) $$ x(t+h)=x(t)+f(x(t))h+\frac{h^2}2f'f+\frac{h^3}6(f''[f,f]+f'^2f)+... $$ As you can see, the difference from the Euler step to the exact step starts with $$ -\frac{h^2}2f'(x(t))f(x(t)) $$

For the implicit Euler step one has to solve $$ u=x(t)+hf(u), ~~~u=x(t)+O(h) $$ Inserting that formula repeatedly into the argument of $f$ and using Taylor expansion gives \begin{align} u&=x+hf(x+hf(u)) \\ &= x+hf(x)+h^2f'(x)f(u)+O(h^3) \\ &=x+hf(x)+h^2f'(x)f(x)+O(h^3) \end{align} This is $$ +\frac{h^2}2f'(x(t))f(x(t))+O(h^3) $$ larger than the expansion of the exact step, as announced the opposite of the error of the implicit Euler method.