Suppose that a real valued function $f$ is continuous at $x_0$. Then, the epsilon delta definition says that for any $\epsilon > 0$, there is a $\delta > 0 $ such that $|f(x_0)-f(x)| < \epsilon $ whenver $|x-x_0| < \delta$.
Based on this definition, how do I show that $f(x_0+h)-f(x_0) = O(h)$? That is, $|f(x_0+h)-f(x_0)| \leq C h$ for some constant $C$ and all $h$ such that $h \leq h_0$ for some $h_0$?
$f$ needs not be differentiable, so I cannot really use the Taylor's theorem here.
On
This is true if $f$ is differentiable at $x_0$: in this case, you have $$ \frac{f(x_0+h)-f(x_0)}{h} \xrightarrow[h\to0]{} f'(x_0) $$ from which $\lvert f(x_0+h)-f(x_0) \rvert = O(h)$.
This is still true if $f$ is only Lipschitz (on a neighborhood of $x_0$): then, there is a constant $C\geq0$ such that $$ \lvert f(x_0+h)-f(x_0) \rvert \leq C \lvert h \rvert $$ from which $\lvert f(x_0+h)-f(x_0) \rvert = O(h)$.
This is false in general: continuity alone is not strong enough an assumption. Rastapopoulos' answer gives the counterexample of the function $\sqrt{\cdot}$ at $x_0=0$ (and this will work also with any $x\mapsto x^\alpha$ for $\alpha\in(0,1)$, for instance); but basically, all you can say from continuity is $$ \lvert f(x_0+h)-f(x_0) \rvert = o(1) $$ which is a much weaker statement than $O(h)$.
The statement is not true. Take for example the function $\sqrt{x}$ on $[0,\infty)$. It is continuous at $0$ but $$|\sqrt{0 + h} -\sqrt{0} | = \sqrt{h},$$ which can't be bounded by $C\,h$ for $h\leq h_0$ for any $C$ and $h_0$.