I am trying to understand the concept of local truncation error and came accross this in my lecture notes:
what I don't understand here is where the term 'O' comes from and what it stands for in the expansion.
Please could someone help me with this.
Thank You
I presume you are familiar with the Taylor expansion of a function, otherwise you should start from there. Your notes, first multiply the expression through out by $h$: $$h\tau = y(x_{i+1})-y(x_i) - hf\left(x_i,y(x_i)\right) \qquad [1]$$ Then they expand $y(x_{i+1})$ (not the whole RHS) into its (infinite) "Taylor series around $x_i$", which means: $$y(x_{i+1}) = y(x_i) + y'(x_i)(x_{i+1}-x_i) + \frac{1}{2!}y''(x_i)(x_{i+1}-x_i)^2 + \frac{1}{3!}y'''(x_i)(x_{i+1}-x_i)^3+ \frac{1}{4!}y''''(x_i)(x_{i+1}-x_i)^4 +... \qquad [2]$$
But $(x_{i+1}-x_i) =h$. Inserting into eq.$[2]$ we have
$$y(x_{i+1}) = y(x_i) + y'(x_i)h + \frac{1}{2!}y''(x_i)h^2 + \frac{1}{3!}y'''(x_i)h^3+ \frac{1}{4!}y''''(x_i)h^4 +... \qquad [3]$$
The expression $O(h^3)$ collects all terms of this expansion, from $\frac{1}{3!}y'''(x_i)h^3$ onwards. It is translated $O(h^3)$ = "terms at most of order $h^3$", meaning, roughly, that the limit of the ratio of the sequence of terms included in $O(h^3)$ over $h^3$ is bounded above (look up the "big O - little o" notation if you are not familiar with that). So we write $$ [3] = y(x_{i+1}) = y(x_i) + y'(x_i)h + \frac{1}{2!}y''(x_i)h^2 + O(h^3) \qquad [4]$$ Insert $[4]$ into $[1]$: $$h\tau = y(x_i) + y'(x_i)h + \frac{1}{2!}y''(x_i)h^2 + O(h^3)-y(x_i) - hf\left(x_i,y(x_i)\right) \qquad [5]$$ Note that $y(x_i)$ cancels out. Also, by the premises we have $y'=f(x,y)$ so these temrs also cancel out. Divide also both sides by $h$ and you are left with $$\tau = \frac{h}{2}y''(x_i) + \frac 1hO(h^3) = \frac{h}{2}y''(x_i) + O(h^2)\qquad [6]$$ ...the second equality by the properties of $O()$. Since in this expression, the $h$ appears raised in the first and in the second power, by the properties of $O()$ we can write $\tau = O(h)$ (i.e. we have to go with the smallest power present), which is translated as "first order" (if we had for example the truncation error of some other method as $\tau^* =O(h^2)$ we would say "it is of second order").