Theorem 1.11
Suppose $S$ is an ordered set with the least-upper-bound property, $B \subset S$, $B$ is not empty, and $B$ is bounded below. Let $L$ be the set of all lower bounds of $B$. Then
$\alpha = \sup(L)$
exists in $S$, and $\alpha = \inf(B)$.
In particular, $\inf(B)$ exists in $S$.
Rudin's definitions for upper/lower bounds and supremum/infimum are given as follows, respectively:
1.7 Definition
Suppose $S$ is an ordered set, $E \subset S$. If there exists a $\beta \in S$ such that $x \le \beta$ for every $x \in E$, we say that $E$ is bounded above, and call $\beta$ an upper bound of E.
Lower bounds are defined in the same way (with $\ge$ in place of $\le$).
$ $
1.8 Definition Suppose $S$ is an ordered set, $E \subset S$, and $E$ is bounded above.
Suppose there exists an $\alpha \in S$ with the following properties:
(i) $\alpha$ is an upper bound of $E$.
(ii) If $\gamma < \alpha$ then $\gamma$ is not an upper bound of $E$.
Then $\alpha$ is called the least upper bound of $E$ [that there is at most one such $\alpha$ is clear from (ii)] or the supremum of $E$, and we write
$\alpha = \sup(E)$.
The greatest lower bound, or infimum, of a set $E$ which is bounded below is defined in the same manner: The statement
$\alpha = \inf(E)$
means that $\alpha$ is a lower bound of $E$ and that no $\beta$ with $\beta > \alpha$ is a lower bound of $E$.
Let $S = \{1, 2, 3, 4, 5 \}$ and $B = \{ 3, 4\}$.
Using the two definitions above, we get that
$B$ is
bounded below by $\{ 1, 2, 3\}$ and
bounded above by $\{4, 5\}$.
Let $L$ be the set of all lower bounds of $B$: $L = \{ 1, 2, 3\}$.
L is
bounded below by $\{ 1 \}$ and
bounded above by $\{3, 4, 5\}$.
Since $\gamma, \alpha \in S$, we get that $\sup(L) = 5$ and $\inf(B) = 1$.
I've gone over this many times and tried to followed Rudin's instructions and definitions precisely. Even so, something is going wrong, and, as I said, I'm trying to follow his text precisely.
I would greatly appreciate it if people could please take the time to clarify this.
EDIT:
For $\sup(L)$, definition 1.8 gives us the following.
(i) $\alpha$ is one of $\{3, 4, 5\}$.
(ii) $\gamma = 3, 4 < \alpha = 5$. Therefore, $\gamma = 3, 4$ is not an upper bound of $L$.
Therefore, we get that $\alpha = 5$ is the least upper bound of $L$.
$\sup(L) \neq 5$: this contradicts Definition 1.8.
Let $\gamma = 4$. We have that $\gamma < 5$, but $\gamma$ is an upper bound of $L$ (as you noted, $4$ is an upper bound of $L$). By definition, $\sup(L)$ must satisfy that whenever $\gamma < \sup(L)$, $\gamma$ is not an upper bound of $L$. $5$ doesn't satisfy this, so $\sup(L)$ can't be $5$.