Teasing apart an explanation of the Central Limit Theorem

96 Views Asked by At

I'm looking at the central limit theorem, and cannot see in the explanation given to me how the average of identical distributions results in the normal distribution.

I am told to consider a sequence of independent and identically distributed variables, and then suppose they have a mean $\mu$ and variance $\sigma^2$.

Then if we take the linear combination of each evenly weighted by the total count:

$$\bar{X} = \frac{X_1+X_2+\cdots+X_n}{n}$$

...we have $E(\bar{X}) = \mu$ and $\operatorname{Var}(\bar{X})=\frac{\sigma^2}{n}$, which in turn means:

$$\bar{X}\sim N\left(\mu,\frac{\sigma^2}{n}\right)$$

While this appears logical, if I use a computer to generate an array of exponentially distributed random variables (such as stats.expon.rvs() from the Python scipy library), then as $n$ in this array increases, a histogram of this data shows an exponential distribution of $\mu=1$ and $\sigma=1$, and not a normal one.

I know that the collection of points in any subrange of the exponential's domain $x = [0, +\infty)$ approaches a normal distribution as the subrange gets smaller and the $n$ in that subrange increases, since the exponential distribution in that subrange provides a mean around which random independent samples in the subrange follow a normal distribution.

But this only applies to collections of samples in subranges, and not to the entire distribution itself.

Unless I'm missing something, it appears averaging any random set of samples from an entire given distribution and then taking a histogram of that set just results in that initial distribution, and not a normal one as mentioned.

2

There are 2 best solutions below

3
On BEST ANSWER

I have good news for you: You are confused.

Firstly, what is a sample? You have $X_1,\ldots,X_n$. That is one sample, not a set of $n$ samples.

If you base a histogram on $X_1,\ldots,X_n$, and $n$ is large, then with high probability, the histogram will look like the density of the "original" distribution.

However, that is only one sample, and it is the whole sample, not the mean of the sample. The sample has a mean, $\bar X=(X_1+\cdots+X_n)/n$. That is the mean of only one sample. One then takes the mean of another sample of $n$ observations. And another. And another. And so on. It is those means that will yield approximately a normal distribution if $n$ is large.

1
On

That is exactly it! You said,

"it appears averaging any random set of samples from an entire given distribution and then taking a histogram of that set just results in that initial distribution".

The law of large numbers is the assumption that the limit of those sample means and variance converges to the actual mean and variance respectively, CLT is that as they do so, the differences fluctuate normally.

See classical CLT from https://en.wikipedia.org/wiki/Central_limit_theorem