So, my textbook gives a proof for the following statement
Every sequence of a compact set $S \subset \mathbb{R}$ has a subsequence which converges in $S$.
Textbook Proof: Assume that every open cover of $S$ has a finite subcovering. Let $\{a_j\}$ be a sequence in $S$. Assume, seeking a contradiction that no subsequence of $\{a_j\}$ converges in $S$. This must mean that for every $s \in S$, there is an $\epsilon_s>0$ such that no element of the sequence satisfies $0<|a_j -s | < \epsilon_s$.
Let $I_s = (s - \epsilon_s, s+ \epsilon_s)$. The collection $C = \{I_s\}$ is then an open covering of $S$, and by our hypotheses, there exists a finite subcovering $I_{s_1}, \ldots, I_{s_k}$ of open intervals that cover $S$. But then $S \subset \bigcup_{j = 1}^k I_{s_j}$ contains no elements of $\{a_j\}$, a contradiction. $\square$
Everything in the first paragraph of the proof makes sense to me, but the construction of the $I_s$ is where things become problematic for me. Although the inequality $0<|a_j -s | < \epsilon_s$ holds for every $s \in S$ (here we can't let $s = a_j$ for example because then we would have $0 < 0$). Similarly, for the $I_s$, we can't let $s = a_j$ because then what the heck would $\epsilon_{a_j}$ be? It seems like the $I_s$ are defined only for all $s \in S-\{a_j\}$. Since none of the $I_s$ contain $a_j$, how can we make the blanket claim that $C = \{I_s\}$ is an open covering of $S$? If $a_j \in \bigcup_{s \in S-\{a_j\}} I_s$, wouldn't $a_j$ necessarily have to be in one of the $I_s$? I'm likely very wrong on one of these points, so some clarification would be greatly appreciated!
You're right that the proof as written is not correct for the reason you've identified. Fortunately, it's relatively easy to fix.
The first sentence of the textbook proof is wrong. What's correct is that there is an $\epsilon_s>0$ such that for some $N_s$ no element of the sequence satisfies $0<|a_j -s | < \epsilon_s$ for any $j \gt N_s$.
The second paragraph then leads to the conclusion that $S \subset \bigcup_{j = 1}^k I_{s_j}$ contains no elements of $\{a_j\}$ for $j \gt \max_j \{N_{s_j}\}$, but that's good enough to establish the contradiction you need to complete the proof.
We can make the blanket claim that $\{I_s\}$ covers $S$ because the points $s$ range over all of $S$.