I am following the proof of the IVT on Wikipedia, and have one point of confusion. In the proof they say
By the properties of the supremum, there exists some $a^* \in (c-\delta, c]$ that is contained in $S$, and so
$$f(c) < f(a^*) + \epsilon \leq u + \epsilon$$ Picking $a^{**} \in (c, c+\delta)$, we know that $a^{**} \not \in S$ because $c$ is the supremum of $S$. This means that $$f(c) > f(a^{**}) - \epsilon > u -\epsilon$$
What bothers me is how this proof is splitting the half intervals. How are we sure that for example $a^*$ exists? If this comes from continuity, how do we not know that such an $a^*$ only exists in the other half of the interval? Likewise for $a^{**}$.
So if $c-\delta < x \le c$ then $|x-c| < \delta$ and $|f(x)-f(c)| < \epsilon$ so $f(c) < f(x) + \epsilon$.
$c-\delta < c$ so $c-\delta$ is not an upper bound of $\sup \{y| f(y) \le u\}$. And $c$ is an upper bound of $\sup \{y| f(y) \le u\}$. That means there must exist a $a^*$ so that $c-\delta < a^* \le c$ where $a^* \in \sup \{y| f(y) \le u\}$.
So $a^* \in (c-\delta, c]$. And we have $a^* \in \sup \{y| f(y) \le u\}$ so $f(a^*) \le u$.
So that means $f(a^*) + \epsilon \le u + \epsilon$.
And so we have
The $f(c) < f(a^*) + \epsilon \le u + \epsilon$.
So for every $\epsilon > 0$ we have
$f(x) \le u + \epsilon$.