Here is my first attempt to prove continuity using epsilon and delta: Prove that $f(x)=\sqrt{x}$ is continuous at $p=4$. Unfortunately my book has just some answers and that isn't one of them. I would like to someone verify my proof:
By definition:
$p-\delta < x < p+\delta\;\;\Rightarrow\;\;f(p)-\epsilon<f(x)<f(p)+\epsilon$
Plug-in the known values:
$4-\delta < x < 4+\delta$
and
$\sqrt{4}-\epsilon<\sqrt{x}<\sqrt{4}+\epsilon$
Then working in the second inequality:
$2-\epsilon<\sqrt{x}<2+\epsilon$
$4-4\epsilon+\epsilon^{2}<x<4+4\epsilon+\epsilon^{2}$
Then let $\epsilon>0$ and set $\delta = 4\epsilon + \epsilon^{2}$, then
$|x-p|<\delta\;\;\Rightarrow\;\;|f(x)-f(p)|<\epsilon$
So $f$ is continuous at $p=4$.
The proof in the OP is absolutely correct. It was mentioned in the comments that we can also choose $\delta = \epsilon$. Then $$|\sqrt{x}-2|=\frac{|x-4|}{\sqrt{x}+2}\leq |x-4|<\delta=\epsilon.$$