In my head integration by parts has always been kind of vague and I just apply it without understanding whats really going on I guess. I.e. if I have the following:
$$\int_{u}^\infty (\ln x - \ln u) \mathrm dF(x) = \int_{u}^\infty \frac{1-F(x)}{x}\mathrm dx$$
I don't know how to handel it. For one I can't deal with the $\mathrm dF(x)$... I know it's like the "change in $F(x)$" and it's the "same" as $f(x)\mathrm dx$ so I write out the left had side like:
$$\int_{u}^\infty (\ln x - \ln u) \ f(x)\mathrm dx$$
and try to do integration by parts like:
$$u=(\ln x -\ln u) \qquad v= 1-F(x)$$
$$u'= \frac{1}{x} \qquad v'= f(x)$$
and then use the IBP formula to get the right hand side, but, for example, I don't really know "why" $v$ is $1-F(x)$ aside from maybe that the range of integration goes from $u$ to $\infty$, so like $ 1-F(x)$ would be $\operatorname{Pr}(X>u)$ sort of. It's all very loosely held together in my head and I usually get by in an exam since I'm fairly certain the answer is right but if I had to explain it, it kind of falls apart. Sort of the hazard of learning by mostly doing dozens of past papers I guess.
Can anyone fill the gaps. I'd appreciate multiple answers if possible (I tend to learn from what multiple sources say and have in common).
Integration by parts when there is no boundary term is often equivalent to some Fubini-type change of order of integration. To apply this general principle in the present case, one can start from the identity, valid for Lebesgue-almost every $x$, $$1-F(x)=P(X>x)=\int_x^\infty dP_X(t)$$ which implies that the RHS of the identity of interest is $$\int_u^\infty\frac{1-F(x)}x\,dx=\int_u^\infty\frac1x\left(\int_x^\infty dP_X(t)\right)dx=\int_u^\infty\left(\int_u^t\frac1xdx\right)dP_X(t)$$ that is, $$\int_u^\infty\frac{1-F(x)}x\,dx=\int_u^\infty\log\left(\frac{t}u\right)\,dP_X(t)$$ and the proof is complete.
The existence of the PDF of $X$ is not required and one only manipulates nonnegative terms in all this hence the identity holds for every random variable, with a PDF or not, whether the resulting integrals are finite or not (if they are not, one arrives at $+\infty=+\infty$, which is true). For an old mse related question, see here.