The epsilon delta definition requires there to be a delta for all epsilon but at https://openstax.org/books/calculus-volume-1/pages/2-5-the-precise-definition-of-a-limit example 2.41 it says:
Prove $\lim\limits_{x \to 2} x^2 = 4$
Without loss of generality, assume $\epsilon \leq 4$ (since $\delta \leq 2 - \sqrt{4 - \epsilon}$), this is allowed because if we can find $\delta>0$ that “works” for $\epsilon \leq 4$, then it will “work” for any $\epsilon>4$ as well. Keep in mind that, although it is always okay to put an upper bound on $\epsilon$, it is never okay to put a lower bound (other than zero) on $\epsilon$.
I don't understand why a delta for a restricted range of epsilon implies there exist a delta for all epsilon as said above.
The definition of a limit says:
So we are only interested in narrowing the epsilon range and seeing whether appropriate deltas exist for smaller and smaller epsilons.
If some delta 'works' for some specified epsilon, we are no longer interested in bigger values of epsilon – the same delta satisfies the condition for them.
Explicitly, if for some positive $\varepsilon_1$ and $\delta_1$ we have that $$x\in(p-\delta_1,p+\delta_1) \implies f(x)\in (q-\varepsilon_1, q+\varepsilon_1)$$ then for any $\varepsilon_2 > \varepsilon_1$ we also have $$x\in(p-\delta_1,p+\delta_1) \implies f(x)\in (q-\varepsilon_2, q+\varepsilon_2)$$ because $$(q-\varepsilon_1, q+\varepsilon_1) \subset (q-\varepsilon_2, q+\varepsilon_2).$$