I'm reading Chapter 11 of Puterman's book on Markov Decision Processes (in particular, about continuous-time Markov processes). There's a lot of notation involved, but I've tried to distill the question. Puterman defines a function $Q(t,j|s,a)$, which, as a simple example, might equal $$ Q(t,j|s,a)=\frac{1}{4}(1-e^{-\mu{t}}) $$ for some $\mu>0$. The function $Q$ is a joint probability distribution in $t\geq0$ and $j\in{S}$ for finite $S$ (in the example above, the product of the CDF of an exponential random variable with a constant). He then writes down the integral $$ \int_0^\infty e^{-\alpha{t}}Q(dt,j|s,a), $$ and asserts that the value of this integral is $<1$. Puterman states "[w]e use $Q(dt,j|s,a)$ to represent a time-differential", but I don't know what this means in the context of integration.
Question 1 What kind of integral is this? Seems like Riemann-Stieltjes or Lebesgue, but I can't tell. I thought it might be strange notation for $$ \int_0^\infty e^{-\alpha{t}}Q(t,j|s,a)dt, $$ but it seems that's not the case (as then the integral can easily be $\geq1$).
Question 2 How do you evaluate such an integral? Is there e.g. a closed-form for the $Q$ defined above?