Law of large numbers without iid assumption

Question

As this is just a reference request, I hope I can get away with being brief. I am simply wondering about generalizations of the classical strong law of large numbers, which states that $$n^{-1}\sum_{i=1}^n X_i \to EX$$ almost surely whenever $(X_n)$ is an iid sequence of integrable random variables.

What can we say about convergence of sample mean to expected value without the iid assumption? In other words, how can we weaken the assumption of iid and still get the desired convergence?

Answer 1

You can think of two well known generalizations. You can weaken the assumption "identically distributed". If for instance $(X_1, X_2, ...)$ are independent and that $X_n \rightarrow_{n \rightarrow \infty} \nu$ in law, then it should not be difficult (assuming the $X_i$'s and $\nu$ to be $L^1$) to prove that: $$a.s. \frac{1}{n} \sum_{i = 1}^n{X_i} \rightarrow_{n \rightarrow +\infty} \int_0^\infty{x \nu(dx) } $$

You can also leverage the independence assumption. For example in the context of Markov Chains or more generally of stochastic processes, the Birkhoff ergodic theorem (when it applies) gives a strong law of large numbers for strongly correlated random variables.

Answer 2

One can relax the independent identically distributed assumption in the following ways.

1. Assume that the sequence $\left(X_i\right)_{i\geqslant 0}$ is strictly stationary, that is, for all $N$, the vectors $\left(X_i\right)_{i=1}^n$ has the same distribution as $\left(X_{i+1}\right)_{i=1}^n$. This implies in particular that $X_i$ has the same distribution as $X_1$ for all $i$. Ergodic theorem tells that if $X_1$ is integrable, then $\sum_{i=1}^nX_i/n\to \mathbb E\left[X_1\mid\mathcal I\right]$ almost surely, where $\mathcal I$ is the $\sigma$-algebra of invariant sets: we represent $(X_i)_{i\geqslant 0}$ as $(f\circ T^i)_{i\geqslant 0}$ where $T$ is measure preserving and $\mathcal I=\{A\mid T^{-1}A=A\}$.

2. An other way to relax the i.i.d. assumption is to work with martingales. In the case of identically distributed case (which is less restrictive than strict stationarity) martingale differences, one needs the assumption of integrability of $\left\lvert X_1\right\rvert\log\left(1+\left\lvert X_1\right\rvert\right) $ (see this paper by J. Elton).

3. Some other useful results on pairwise dependent random variables can be found in this paper by V. Korchevsky and the references therein.