Why aren't the strong LLNs and CLT contradicting each other?

Question

Given $n$ i.i.d. random variables $\{X_1, X_2, \dots , X_n\}$, each with mean $M$ and variance $V$, both strong and week LLNs seem to say that the average of the $n$ random variables, $S_n = \frac{X_1 + X_2 + \dots + X_n }{n}$, approaches $M$, as $n \to \infty$. The CLT seems to say that, as $n \to \infty$, the distribution of this average $S_n$ approaches a normal distribution with mean $M$ and variance $V$.

The problem I'm having is that it seems like the distribution of the average should converge to something like a discrete variable with a PMF like $1$ at $M$ and $0$ everywhere else. This is because the strong LLN says the average must be $M$, as $n$ approaches infinity. Instead, the normal distribution given by the CLT seems to say that there's a chance of the average not being $M$, as $n$ approaches infinity, which seems to contradict the strong LLN.

Where's the flaw in my reasoning?

Answer 1

The problem is that you left out the scaling. The average does converge to a constant: what converges to a normal distribution is $$\left(\text{average} - M\right)\sqrt{n}$$

Answer 2

Heres my 5 cents.

Central limit theorem states that:

If we have $X_1,X_2,\dots$ iid with mean $\mu$ and variance $\sigma^2$ then $$U_n=\frac{1}{\sqrt{n\sigma ^2}} \sum _{i=1}^n (X_i - \mu) \to N(0,1)$$ in distribution.

Law of large numbers states that:

If $X_1,X_2,\dots$ are i.i.d with mean $\mu$ then $$S_n=\frac{1}{n}\sum_{i=1}^n X_i \to \mu $$ a.s.

Returning to the CLT we can see it also shows that (since $\frac{1}{\sqrt n}=\frac{\sqrt n}{\sqrt n \sqrt n}$): $$\sqrt{n}\cdot (\frac{1}{n} \sum _{i=1}^n (X_i - \mu)) \to N(0,\sigma ^2)$$

So you can see that CLT "works" a bit slower which is where the difference lies.

Convergence a.s. is much stronger than convergence in distribution, so you are right - had the scaling been the same it would imply convergence in distribution to a degenerate, that is almost surely constant random variable. In the case above $S_n \to X$ in distribution where $P(X=\mu)=1$

Maybe you can even now use CLT to say something about how fast the LLN convergence is?