Here are two statements that the textbook considers:
(1) For any two real numbers $a, b$
$$a \leq b \quad iff \quad a < b + \epsilon, \quad \forall \epsilon > 0$$
(2) Given an upper bound $s$ of a subset $A \subseteq \mathbb{R},$
$$s = \operatorname{sup}A \quad iff \quad \exists a \in A \quad s.t. \quad s - \epsilon < a, \quad \forall \epsilon > 0.$$
However, is it not the case that, by the first statement, if
$$s - \epsilon < a, \quad \forall \epsilon > 0,$$
then
$$s \leq a.$$
However, as $s$ is given to be an upper bound of the set $A,$ it is also the case that
$$s \geq a.$$
Therefore, it is necessarily the case that
$$s = a.$$
However, it is obviously not "necessarily" the case that the supremum of a set $A$ must be included in that set $A$ (that is, that the supremum of a set $A$ must be equal to some number in that set $A$).
Where am I confused? Or, where is Abbott confused?
Your attempt to rephrase Abbott resulted in an ambiguous statement:
$$s = \sup A \quad \mathrm{iff} \quad \exists a \in A \quad \mathrm{s.t.} \quad s - \epsilon < a, \quad \forall \epsilon > 0.$$
Does this mean $$s = \sup A \quad \mathrm{iff} \quad ((\exists a \in A \quad \mathrm{s.t.} \quad s - \epsilon < a), \quad \forall \epsilon > 0), \tag1$$
or does it mean $$s = \sup A \quad \mathrm{iff} \quad (\exists a \in A \quad \mathrm{s.t.} \quad (s - \epsilon < a, \quad \forall \epsilon > 0))? \tag2$$
Indeed $(2)$ would be incorrect in this context, because it is indeed equivalent to $s = \sup A \iff (\exists a \in A:s \leq a).$ Note that it is true that $\sup A \leq \max A$ when $\max A$ exists (that is, when $A$ has a greatest element), but the idea of the supremum is that it is defined even when $A$ does not have a greatest element.
But $(1)$ is OK (apart from the fact that is an abominable mangling of the notation of predicate logic). Note that if we write $$a \leq b \iff (\forall \epsilon > 0 : a < b + \epsilon),$$ we fix the numbers $a$ and $b$ first, and then we say that a certain inequality involving those two already-chosen numbers must be true for every value of $\epsilon$ that we might put in that inequality. But in the right-hand side of $(1),$ which (more conventionally written) is $$\forall \epsilon > 0 : \exists a \in A : s - \epsilon < a,$$ the $\forall$ symbol means the rest of the formula has to be evaluated for every possible positive value of $\epsilon,$ but each time we choose $\epsilon$ first and then say something about $a$ and $s.$ The way this works when there is no greatest element in $A$ is that we are allowed to choose different values of $a$ for different values of $\epsilon,$ and indeed we will have to do so.