I'm looking for a specific counterexample for the weak law of large numbers. That is, I want a sequence of random variables with same finite expected value $\mu$. These random variables must each have finite variance $\sigma_i^2$.
I guess that the $\sigma_i^2$ should be unbounded for this to work. Is there a known counterexample for this?
I'd also appreciate a book reference for me to learn a bit more!
Let $X_0, X_1, X_2, \dots$ be independent with $X_n = \pm 4^n$ each with probability $1/2$. Then $E[X_n] = \mu = 0$ for all $n$. Letting $S_k = X_0 + \dots + X_k$, we have $$|S_{k-1}| \le \sum_{n=0}^{k-1} 4^n = \frac{1}{3} (4^k - 1)$$ so $$|S_k| \ge ||X_k| - |S_{k-1}|| \ge 4^k - \frac{1}{3} (4^k - 1) \ge \frac{2}{3} 4^k.$$ In particular, $|S_k/k| \to \infty$ almost surely.