Understanding proof of $\mathbb{E}[X^n] = \int_{0}^\infty n x^{n-1} (1 - F(x)) dx$

Question

For a nonnegative random variable $X$ with CDF $F$ and $n \geq 1$,

$\begin{align*} \mathbb{E}[X^n] &= \int_{0}^\infty x^n d F(x)\\ &= \int_{0}^\infty \left ( \int_{0}^x n y^{n-1} dy \right ) d F(x) \\ &= \int_{0}^\infty \int_{y}^\infty ny^{n-1} dF(x) dy\\ &= \int_{0}^\infty n y^{n-1}(1 - F(y)) dy \end{align*}$

1. I have not seen the notation in the first equality before. All I know is $\mathbb{E}[X^n] = \int_{0}^\infty x^n f_X(x) dx$ but I know the pdf $f_X$ does not necessarily exist. What does $dF(x)$ in the first equality mean?
2. How to go from second to third equality? Interchanging the order of integrals should give $\int_{0}^x \int_{0}^\infty$; how to understand the new limits?
3. The 4th equality apparently uses the result $1 - F(y) = \int_{y}^\infty dF(x)$ which I don't understand because I don't know what $dF(x)$ means in the first place.

Answer 1

First a few things about CDF. Let $X$ be any random variable and $F$ be the CDF of $X.$ What we know about $F$ is that $F$ is increasing, right-continuous, and $F(-\infty)=0=1-F(\infty).$ Since $F$ is an increasing function; writing $dF(x)$ makes perfect sense, that is, we can integrate (Reimann-Stiltjes integral) with respect to $F,$ and it is not hard to show that if $X$ has a density $f_X$ then $dF(x)=f_X(x)dx.$

Now, the first equality is nothing but the definition. If you want to be more rigorous: Let $X$ be a random variable on some probability space $(\Omega, \mathcal{F}, \mu),$ and $h:\mathbb{R}\to \mathbb{R}$ be any function such that $h(X)\in L^1(\mu),$ then $\mathbb{E}(h(X)):=\int_{\Omega}h(X)(\omega)d\mu(\omega).$ Now a simple change of variable tells us that $\mathbb{E}(h(X))=\int_{\mathbb{R}}h(x)d\mu_X(x)$ where $\mu_X$ is the distribution of $X,$ in other words, $\mu_X$ is a measure on $\mathbb{R}$ defined as $\mu_X(A):=\mu(X^{-1}(A)).$ You can take $dF(x)$ to mean the measure $d\mu_X,$ in fact for any measure on radon measure $\mu$ on $\mathbb{R}$ one can associate a distribution function $F_{\mu}$ such that the integral with respect to $\mu$ is same as the Stiltjes integral with respect to the distribution function $F_{\mu}.$

With all this technicality in mind, now the first equality is just the definition. The second equality is a simple verification, you can check. The third equality is Fubini's theorem (to be honest Tonelli's theorem, and this is where we use that $X$ is non-negative), and hence you can change the order of integration. And, once you get around what $dF(x)$ means, you know that your third statement is indeed true, and you know why.