We know that the definition of a limit is $\forall\epsilon > 0 \space \exists\delta >0, 0 < |x-c| < \delta \space \to \space |f(x) - L| < \epsilon$
However, won't the definition stay the same if we were to write $\forall\epsilon > 0 \space \exists\delta \ge0, \space0 < |x-c| < \delta \space \to \space |f(x) - L| < \epsilon$, since for each epsilon there has to be some delta that works. Therfore, if since $\delta \ge 0$, it follows that $\delta > 0$?
Also, shouldn't that be the same for $\forall\epsilon > 0 \space \exists\delta >0, 0 < |x-c| < \delta \space \to \space |f(x) - L| \le \epsilon$ as well?
You are allowed to write $\exists\delta \ge0$ instead of $\exists\delta>0$ since you see in the definition of the limit that $\delta$ cannot take the value $0$ because that would produce contradiction in definition.
You shuldn´t write $\epsilon \geq 0$ instead of $\epsilon >0$ since for $\epsilon =0$ you would have that $f$ is necessarily constant function and you want that our definition of the limit covers more general class of functions than just constant functions.