How do I arrive at this Taylor series expansion for the given function?

Question

This is taken from my textbook on numerical analysis/scientific computing:

I don't quite understand the explanations here and was hoping someone could help break this down for me. I am able to get the first two terms by simply rearranging the given approximation:

$f(x+h) = h*f'(x) + f(x) = f(x) + f'(x)h$

But the third term, and the explanation in the text that follows, seems to appear out of nowhere.

From Taylor series, I recognize it as the natural third term that would arise in approximating a function, but what's with the theta and the stuff about $M$?

Answer 1

First, note that $$f(x+h) \simeq h*f'(x) + f(x) = f(x) + f'(x)h$$ is an approximation, not an equality.

The error of the estimate can be written explicitely by the Lagrange Remainder, a more general version of the Mean Value Theorem. This result is exactly what they use here.

Answer 2

It's just Taylor's theorem with Lagrange remainder: $$f(x+h)=\sum\limits_{j=0}^n\frac{h^j}{j!}f^{(j)}(x)+\frac{h^{n+1}}{(n+1)!}f^{(n+1)}(\theta),$$ where $\theta$ is between $x$ and $x+h$ (for a sufficiently regular function).