My notes contain the following notation
$$E(\theta\mid X) = \int \theta \, df(\theta\mid X) = \int \theta f(\theta\mid X) \, d\theta$$
The first integral looks like a Riemann–Stieltjes integral, although this sequence of equalities does not look correct to me. I thought that the general relation is (non-rigorously)
$$\int f(y)\,dg(y) = \int f(y)g'(y)\,dy$$
which indicates that $df(\theta \mid X)$ in the first line should instead be $dF(\theta \mid X)$, the cumulative density distribution function. Is there an error in the notes or am I missing something in this probabilistic interpretation?
Yes, it is meant to be:
$$\begin{align} \mathsf E(\theta \mid X) & = \int \theta \;\operatorname d F(\theta\mid X) \tag{capital F} \\ & = \int \theta \;f(\theta \mid X) \operatorname d \theta \tag{lowercase f} \end{align}$$