A generalization of the law of large numbers.

143 Views Asked by At

Let $X\subset \mathbb{R}^m$ be compact. Let $X_1,\dots,X_n \sim_{iid}\mathcal{U}(X)$. By the law of large numbers, we have $P(\lim_{n\to \infty}\frac{1}{n}\sum_{i=1}^n(f(X_i)-E[f(X_i)])=0)=1$ for all $f\in C(X)$, since $X$ is compact and our random-variables are iid.

I was wondering, if the following statement holds: $$ P\left(\forall f\in C(X):\lim_{n\to \infty}\frac{1}{n}\sum_{i=1}^n(f(X_i)-E[f(X_i)])=0\right)=1$$ I would assume it doesn't, but I am not able to find a counterexample.

1

There are 1 best solutions below

0
On BEST ANSWER

This is true for any $X$ which is a compact metric space (need not be Euclidean as you have assumed). The key fact that you will use here is that $C(X)$ is separable, a result which follows from application of Stone-Weierstrass Theorem and Uryshon's Lemma.

Let $$ A_f := \left\{ \omega \in \Omega \mid \dfrac{1}{n} \sum_{i=1}^n \left[f(X_i(\omega)) - \mathbb{E}f(X_i) \right] \to 0 \right\}, \; \forall \; f \in C(X).$$ $A_f$ is clearly an event and by SLLN, $\mathbb{P}(A_f)=1$ for all $f \in C(X)$.

Let $\mathcal{M}$ be a countable dense subset of $C(X)$. Then $\mathbb{P}(A)=0$, where $A:= \bigcap_{f \in \mathcal{M}} A_f.$ I claim that $A= \bigcap_{f \in C(X)} A_f$. That will complete the proof.

To see why this claim is true, note that obviously $A \supseteq \bigcap_{f \in C(X)} A_f$. Now take any $\omega \in A$ and fix $f \in C(X)$. All I need to show is that $$ \sum_{i=1}^n \left[f(X_i(\omega)) - \mathbb{E}f(X_i) \right] \to 0.$$ Fix any $\varepsilon >0$. Since $\mathcal{M}$ is dense in $C(X)$, you can get $g \in \mathcal{M}$ such that $||f-g||_{\infty} \leq \varepsilon$. Then \begin{align*} \Bigg \rvert \sum_{i=1}^n \left[f(X_i(\omega)) - \mathbb{E}f(X_i) \right]\Bigg \rvert & \leq \Bigg \rvert \sum_{i=1}^n \left[f(X_i(\omega)) - g(X_i(\omega)) \right]\Bigg \rvert + \Bigg \rvert \sum_{i=1}^n \left[g(X_i(\omega)) - \mathbb{E}g(X_i) \right]\Bigg \rvert \\ & \hspace{1 in} + \Bigg \rvert \mathbb{E}(f(X_1))-\mathbb{E}(g(X_1))\Bigg \rvert \\ & \leq 2\varepsilon + \Bigg \rvert \sum_{i=1}^n \left[g(X_i(\omega)) - \mathbb{E}g(X_i) \right]\Bigg \rvert \to 2\varepsilon. \end{align*} Since, $\varepsilon$ is arbitrary this proves the claim.