Suppose $(\Omega, \mathcal{F}, \mathbb{P})$ is a probability space, and let $X_k: \Omega \to \mathbb{R}$ be a sequence of identically distributed random variables, with expectation \begin{equation} \mathbb{E} X_k = \mu \in [0,1],\quad \forall k. \end{equation} Define the sample average, \begin{equation} \bar{X}_n := \frac{1}{n}\sum_{k = 1}^{n}X_k. \end{equation}
Can we say anything about the convergence of $\bar{X}_n$? Can we say $\bar{X}_n$ converges (to something, not necessarily $\mu$) as $n \to \infty$? What can go wrong?
Of course, if the $X_k$'s are, in addition, independent, then the Strong Law of Large Numbers (SLLN) says the sample average converges to $\mu$, \begin{equation} \bar{X}_n \to \mu,\quad \mathbb{P}-\text{a.s.} \end{equation} But I am curious about dropping the independence criterion. A simple counterexample removing independence for the SLLN would be to take $X_k = X$ for all $k$, for some $X$ with $\mathbb{E} X = \mu$. But in this case we still have (trivially) convergence of $\bar{X}_n$.
Let $\Omega = \{0,1\}^\mathbb{N}$ with the Borel sigma-algebra and $\mathbb{P}$ be the $\frac{1}{2}-\frac{1}{2}$ product measure. Let $X(\omega_1,\omega_2,\dots)$ be $0$ if $\omega_1 = 0$ and $1$ if $\omega_1 = 1$. Let $Y(\omega_1,\omega_2,\dots)$ be $1$ if $\omega_1 = 0$ and $0$ if $\omega_1 = 1$. Let
$X_1 = X$
$X_2,X_3 = Y$
$X_4,X_5,X_6,X_7 = X$
$X_8,\dots,X_{15} = Y$.
Etc. The point is that for any $\omega$, the sums $\frac{1}{N}\sum_{n \le N} Y_n(\omega)$ won't converge for any $\omega$.