In Baby Rudin, $\textbf{Theorem 6.9}$ states that $\textit{If f is monotonic on [a,b], and if $\alpha$ is continuous on [a,b], then $f \in \mathcal{R}(\alpha).$}$
To prove this, Rudin mentions that we can choose a partition such that \begin{align*} \Delta\alpha_i = \frac{\alpha(b) - \alpha(a)}{n} \end{align*} for $1 \leq i \leq n.$ And he adds that we can do this since $\alpha$ is continuous (from Theorem 4.23). I might be misunderstanding something about continuity, but do we need $\alpha$ to be continuous in order to use this partition in the proof? Can't we use the same partition as long as $\alpha$ is defined $\forall x \in [a,b]$?
I would greatly appreciate clarification on this.
Yes, we do require $\alpha$ to be continuous in order to be able to pick $\Delta \alpha_i = \frac{\alpha(b) - \alpha(a)}{n}$. Let's look into this in more detail.
We want to pick $x_0, x_1, ..., x_n$ such that $a = x_0 \leq x_1 \leq ... \leq x_n = b$ and such that $\Delta \alpha_i = x_{i + 1} - x_i = \frac{\alpha(b) - \alpha(a)}{n}$ for all $i \in \mathbb{N}$ s.t. $0 \leq i < n$.
In other words, we must choose $a = x_0 \leq ... \leq x_n = b$ such that $\alpha(x_i) = \frac{i \alpha(b) + (n - i) \alpha(a)}{n}$ for all $i$.
We can get a choice of the $x_i$ as follows:
Pick $x_0 = a$. For each $i$ such that $0 \leq i < n - 1$, suppose we've already picked $x_i$ such that $\alpha(x_i) = \frac{i \alpha(b) + (n - i) \alpha(a)}{n}$. Note that $\frac{(i + 1) \alpha(b) + (n - i - 1) \alpha(a)}{n}$ is between $\alpha(x_i)$ and $\alpha(b)$. Thus, by the intermediate value theorem (continuity used here), there is some $x_{i + 1}$ between $x_i$ and $b$ such that $\alpha(x_{i + 1}) = \frac{(i + 1) \alpha(b) + (n - i - 1) \alpha(b)}{n}$.
Finally, pick $x_n = b$.
Note that the critical step is the use of the intermediate value theorem. In principle, we could use a non-continuous function which satisfies the intermediate value property (for example, any function which is not continuous but is the derivative of some function), but there are few "natural" examples of this.