How to understand the function $o(h) + o(h) = o(h)$?

Question

On page 150 of the book Introduction to Mathematical Statistics, there's the following passage:

Let the symbol $o(h)$ represent any function such that $lim_{h→0}[o(h)/h] = 0;$ for example, $h^2 = o(h)$ and $o(h) + o(h) = o(h)$.

I can understand how $h^2 = o(h)$ fits the description. But I couldn't understand the function $o(h) + o(h) = o(h)$.

Answer 1

Big- and little-O notation can be confusing because it violates everything we know about using variables in equations, since each instance of $o(h)$ refers to a (potentially) different function and the quantification (i.e. 'for all' vs. 'there exists') has to be inferred from context.

When we write an equation involving $o(h)$ terms, you should read it as saying that each $o(h)$ can be replaced by $f(h)$ for some function $f$ satisfying $\lim_{h \to 0} \frac{f(h)}{h} = 0$, and that the function $f$ might be different each time.

So in your case, writing $o(h)+o(h)=o(h)$ means:

For all functions $f,g$ such that $\lim_{h \to 0} \frac{f(h)}{h} = 0$ and $\lim_{h \to 0} \frac{g(h)}{h} = 0$, there exists a function $k$ such that $\lim_{h \to 0} \frac{k(h)}{h} = 0$ and $f(h)+g(h)=k(h)$.

(Note that $f$ and $g$ are universally quantified and $k$ is existentially quantified.)

Proving this statement is now easy using elementary facts about limits of functions: just define $k(h)=f(h)+g(h)$ and prove that $\lim_{h \to 0} \frac{k(h)}{h} = 0$.

Answer 2

The equality $o(h) + o(h) = o(h)$ can be expanded into the following statement: let $f(h),g(h)$ be functions. If $f = o(h)$ and $g = o(h)$, then $f + g = o(h)$.

We can see that this is the case from definitions. In particular, we note that $$ \lim_{h \to 0} \frac{f(h) + g(h)}{h} = \lim_{h \to 0} \frac{f(h)}{h} + \lim_{h \to 0} \frac{g(h)}{h} = 0+0 = 0 $$

Answer 3

Let $o_1$ and $o_2$ be two functions satisfying $\lim_{h\to0}\frac{o_i(h)}h=0$. Then $\lim_{h\to0}\frac{(o_1+o_2)(h)}h=\lim_{h\to0}\frac{o_1(h)}h+\lim_{h\to0}\frac{o_2(h)}h=0+0=0$, so $o_1+o_2$ also satisfies this condition.