LLN and rates in terms of Stieltjes integral

Question

I am learning about Stieltjes integrals. Written in terms of a Stieltjes, the law of large numbers, $\frac1n\sum_{i=1}^n f(X_i)\to E(X)$ as $n\to\infty$, looks like: $$ \int_{-\infty}^{\infty}f(x)dF_n(x)\to \int_{-\infty}^{\infty}f(x)dF(x) $$ or $$ \int_{-\infty}^{\infty}f(x)d(F_n-F)(x)\to 0 $$ I am wondering 1. Can this be proven just using standard Stieltjes integral material and perhaps the additional fact about the uniform convergence $F_n-F\to 0$? 2. Can you actually get the usual order of an IID average, $\frac1n\sum_{i=1}^n f(X_i)- Ef(X)=O(n^{-1/2})$ just using Stieltjes theory and the fact that $F_n-F$ is uniformly $O(n^{-1/2})$? I did not specify the mode of convergence for the LLN above since I don't know how much can be proven from this route.

Answer 1

Answer to 1)

We know that $\sup_x |F_n(x) - F(x)| \to 0$ by Glivenko–Cantelli theorem. Suppose that we "forgot" about the fact that $F_n$ corresponds to a sample from i.i.d. So lets condiser a sequence of c.d.f. $F_n(x)$ and $F(x)$. We know, that $\sup_x |F_n(x) - F(x)| \to 0$. Does it follow that $\int_{-\infty}^{\infty}f(x)dF_n(x)\to \int_{-\infty}^{\infty}f(x)dF(x) $?

No. Suppose that $f(x) = x$ and $F_n$ corresponds to $n \cdot Bern(1/n)$, so $F_n = 1$ if $x \ge n$, $F_n = 0$ if $x < 0$ and $F_n= 1-\frac{1}n$ otherwise. Hence $\int_{-\infty}^{\infty}f(x)dF_n(x) = \mathbf{E} ( nBern(1/n) ) = n\cdot \frac1n = 1$. Put $F(x) = 1$ if $x \ge 0$ and $F(x) = 0$ otherwise. $F(x)$ is a c.d.f. of constant $0$. Hence $\int_{-\infty}^{\infty}f(x)dF(x) = \mathbf{E} 0 = 0$ and $\sup_x |F_n(x) - F(x)| \to 0$. Q.e.d.

Answer to 2) We know that $\sqrt{n }\sup_x |F_n(x) - F(x)| \overset{d}{\to} \xi \sim \sup_t W_0(F(t))$ (see "Convergence of probability measures", Billingsley). Suppose that we "forgot" about the fact that $F_n$ corresponds to a sample from i.i.d. So lets condiser a sequence of c.d.f. $F_n(x)$ and $F(x)$. We know, that $\sqrt{n }\sup_x |F_n(x) - F(x)| \overset{d}{\to} \xi$. Does it follow that $\int_{-\infty}^{\infty}f(x)dF_n(x) - \int_{-\infty}^{\infty}f(x)dF(x) = O(n^{-\frac12})?$

No. Consider the previous counterexample. We have $\sqrt{n}\sup_x |F_n(x) - F(x)| = \frac{1}{\sqrt{n}}$ and hence $\sup_x |F_n(x) - F(x)| = O(\frac1{\sqrt{n}})$ and $\int_{-\infty}^{\infty}f(x)dF_n(x) - \int_{-\infty}^{\infty}f(x)dF(x) = 1 \ne O(n^{-\frac12})?$.