Confusion about Central limit and large number theorem

Question

Intuitively, the large number theorem says the average converges to the mean with probability 1, outside there are almost zero. But central limit says it is a bell curve, but the bell curve doesn’t vanish sharply outside the mean. Is that a contradiction?

Answer 1

It's a matter of scales. The law of large numbers says that the sample mean goes to the population mean. The central limit theorem says that:

1. Typical deviations between the sample mean and the population mean scale like $n^{-1/2}$ for large $n$
2. Multiplying these (signed) deviations by $n^{1/2}$ results in a normal distribution whose variance is the population variance (or rather, this happens asymptotically).

Since $n^{-1/2}$ goes to zero, there is no contradiction between the two results. In fact the central limit theorem implies the weak law of large numbers.

There is in fact a third class of results, called "large deviation" results, which give estimates of the probability that the sample mean deviates from the population mean by more than some fixed amount. These estimates are more accurate than the corresponding central limit theorem estimates.

Answer 2

Of course two of the most famous theorems in probability do not contradict one another.

The central limit theorem says that $$ \frac{\overline X-\mu}{\sigma/\sqrt{n}} \to_D N(0,1)$$ (where $\to_D$ means convergence in distribution) which roughly means that as $n$ gets large, the distribution of the sample mean looks like $$\overline X \sim N(\mu, \sigma^2/n).$$

Notice that as $n$ gets large, the variance goes to zero. So this does not conflict intuitively with the law of large numbers, which says that the we have $\overline X\to \mu$ almost surely (or in probability in the case of the weak law of large numbers).