I need some help figuring this one out:
Is $(1 + O(h))^{-1}$ also equal to $1 + O(h)$?
I know that $$\lim_{h \to 0} \frac{O(h)}{h} = C$$ for some $C \in \Bbb R$.
I can show that $$ \frac{1}{1 + O(h)} - 1 = O(h).$$
We have $$ \lim_{h \to 0} \frac{1}{h}\left(\frac{1}{1 + O(h)} - 1\right) = \lim_{h\to 0} \frac{1}{h}\frac{-O(h)}{1 + O(h)} = \lim_{h\to 0}-\frac{C}{1+O(h)}.$$
First things first, $f(h)=O(h)$ when $h\to 0$ means $|f(h)|\le C|h|$ when $h$ is in some neighbourhood of $0$. The condition you are citing ($\lim_{h\to 0}\frac{f(h)}{h}=C$) is just a sufficient condition for that to happen.
Still, the statement you want to prove is true: if $|f(h)|\le C|h|$ in some neighbourhood of $0$, then:
$$|(1+f(h))^{-1}-1|=|f(h)||(1+f(h))^{-1}|\le C(1+\epsilon)|h|$$
in some (potentially smaller, depending on your choice of $\epsilon\gt 0$) neighbourhood of $0$: the factor $1+\epsilon$ comes from approximating $(1+f(h)^{-1})\to 1$ when $h\to 0$.
Therefore, $(1+f(h))^{-1}-1=O(h)$ (take $C(1+\epsilon)$ as the constant multiplier), i.e.$(1+f(h))^{-1}=1+O(h)$.