I am stuck on the following question: what is the necessary and sufficient condition for an increasing function $ \alpha $ so that $ \int_a^b \alpha d \alpha $ exist?
This is probably an easy question but I'm stuck trying to show this from definition. I tried to show that given any $ \epsilon $, there is some partition $ P $ such that $ U(P, \alpha) - L(P, \alpha) < \epsilon $. Clearly, if $ P = ( x_0, \dots, x_n ) $, then we have $ U - P = \sum_{i=1}^n (\alpha(x_i) - \alpha(x_{i-1}))^2 $ since the supremum and the infimum of $ \alpha $ in an interval $ [x_{i-1}, x_i ] $ is simply $ \alpha(x_i) $ and $ \alpha(x_{i-1}) $ respectively. But I'm not sure what to do next? Can someone give a suggestion?
Any monotone increasing function has at worst countably many jump discontinuities. However, a Riemann-Stieltjes integral $\int_a^b f \, dg$ will fail to exist if $f$ and $g$ are discontinuous from the right or left at the same point. This claim is proven below.
Thus, the integral $\int_a^b \alpha \, d \alpha$ exists if and only if $\alpha$ is continuous.
In this case, you can follow your initial approach to produce a sufficiently fine partion so that upper and lower sums differ by less than $\epsilon$, using a slight modification of the proof for Riemann integrals. Otherwise just take as granted the standard theorem of integrability for continuous functions. The challenging part here is to show that Riemann-Stieltjes integrability fails if the integrand and integrator share common discontinuities.
Also, if the integral exists exists we can use integration by parts to immediately find the value
$$\int_a^b \alpha \, d \alpha = \alpha(b)^2 - \alpha(a)^2- \int_a^b \alpha \, d \alpha \\ \implies \int_a^b \alpha \, d \alpha = \frac{1}{2}(\alpha(b)^2 - \alpha(a)^2)$$
Proof of claim
The Riemann-Stieltjes integral does not exist if $f$ and $g$ share a common discontinuity point.
Suppose $g$ is monotone increasing and WLOG $f$ and $g$ are discontinuous from the right at $\xi \in (a,b).$ Consider any partition $P = (x_0,x_1, \ldots, x_{i-1},\xi, x_i, \ldots, x_n)$ with $\xi$ as a partition point and $x_i - \xi = \delta_i$
There exists $\epsilon > 0$ such that for every $\delta > 0$ (including $\delta_i$), there are points $y_1, y_2 \in (\xi, \xi + \delta)$ such that $|f(y_1) - f(\xi)| \geqslant \epsilon$ and $|g(y_2) - g(\xi)| \geqslant \epsilon$.
Then we have
$$U(P,f,g) - L(P,f,g) \geqslant \epsilon^2,$$
since $g(x_i) - g(\xi) \geqslant g(y_2) - g(\xi) \geqslant \epsilon$ and $\sup_{x \in [\xi,x_i]} f(x) - \inf_{x \in [\xi,x_i]} f(x) \geqslant \epsilon$
Therefore, the Riemann criterion is not satisfied and $f$ is not RS integrable with respect to $g$ on $[a,b]$.