Weak Law of Large Numbers

Question

The Weak Law of Large Numbers is often stated with the iid assumption for the underlying RV's. However, I have seen the independence assumption being diluted to the "uncorrelatedness" assumption (e.g., Durrett, Sect. 1.5, 3rd ed). I also vaguely remember (from Billingsley's text?) that the RV's do not even have to be identically distributed. I will appreciate, if someone could shade light on the most general (the least restrictive) form of the WLLN--including the relaxation of the two above-mentioned requirements. Thank you.

Answer 1

Here is a set of conditions that you might be interested in.

Suppose $X_1, X_2, ..., X_n$ are pairwise independent, with $E[X_i]=\mu$, $Var(X_i)=\sigma^2$ for each $i$.

We can prove a similar result to the Weak Law of Large Numbers using Chebyshev's inequality.

$$P\left(|\frac{1}{n}\sum_{i=1}^nX_i - \mu| \geq \epsilon\right) = P\left(|\frac{1}{n}\sum_{i=1}^n(X_i - \mu)| \geq \epsilon\right) = P\left(|\sum_{i=1}^nX_i - \sum_{i=1}^nE[X_i]| \geq n\epsilon\right) \leq \operatorname{Var}\left(\sum_{i=1}^nX_i\right)/(n\epsilon)^2$$

You can calculate $\operatorname{Var}(\sum_{i=1}^nX_i)/(n\epsilon)^2$ using $\operatorname{Var}(X)=E[X^2] - E[X]^2$. The result is $\sum_{i=1}^n \operatorname{Var}(X_i) = n\sigma^2$.

Therefore,

$$P\left(|\frac{1}{n}\sum_{i=1}^nX_i - \mu| \geq \epsilon\right) \leq \frac{\sigma^2}{n\epsilon^2}$$

As $n \rightarrow \infty$, $P(|\frac{1}{n}\sum_{i=1}^nX_i - \mu| \geq \epsilon) \rightarrow 0$. And we have a similar result to WLLN.