Creating an Example to Disprove the Central Limit Theorem?

Question

In class, we are always told that for the Central Limit Theorem to be applicable, observations have to be IID (Independent and Identically Distributed). However, we are not always told why this IID condition is so important for the Central Limit Theorem.

This being said, I am trying to create an example where the IID condition is not met and thus show myself why it is required.

Part 1: The first thing that comes to mind is an Autoregressive Process as by definition AR Processes are said not to be IID. For instance, suppose we have an AR(1) Process:

$$y_t = \phi y_{t-1} + \epsilon_t$$

Based on this AR(1) process, I know the following:

• $E(\epsilon_t) = 0$
• $E(\epsilon_t^2) = \sigma^2$
• $E(\epsilon_t\epsilon_s) = 0$
• $Var(y_t) = \frac{\sigma^2}{1-\phi^2}$
• $E(y_t) = \phi E(y_{t-1}) = 0$

Part 2: As for the Central Limit Theorem, I know that in when $n$ is large, any Random Variable behaves as:

$$\frac{\bar{x} - E(X)}{\sqrt{\frac{Var(X)}{n}}} \approx N(0,1)$$

Part 3: Putting this all together, I would now show that the above AR(1) Process DOES NOT converge to a Standard Normal Distribution:

$$\frac{Y_t - E(Y_t)}{\sqrt{\frac{Var(Y_t)}{n}}} = \frac{Y_t - E(Y_t)}{\sqrt{\frac{\sigma^2}{1-\phi^2}\frac{1}{n}}} \not\approx N(0,1) $$

However, I am not sure if I am doing this correctly for the AR(1) Process and have in fact shown that in the absence of IID, the Central Limit Theorem is not necessarily valid.

In general, can someone please show me an example where the IID condition is not met and as a result the Central Limit Theorem does not apply?

Thanks!

Note: I am aware that there are versions of the Central Limit Theorem that do not require the IID Condition (e.g. https://en.wikipedia.org/wiki/Central_limit_theorem#Lyapunov_CLT, https://en.wikipedia.org/wiki/Lindeberg%27s_condition) - however, I am specifically interested in constructing an example that shows why the Classic Central Limit Theorem requires the IID condition.

Answer 1

A more mundane (and probably less entertaining) answer than the escapades of Prof Scatterbrain.

Independent but not identical. Suppose $X_k, k \geq 2$ is given by

$$ X_k = \begin{cases} -1 & \text{with probability $\frac{1}{1+k^2}$} \\ 0 & \text{with probability $1-\frac{1}{k^2}$} \\ k^2 & \text{with probability $\frac{1}{k^2+k^4}$} \end{cases} $$

Then each $X_k$ has mean $0$ and variance $1$, but

\begin{align} \lim_{n \to \infty} P\left(\sum_{k=2}^n X_k = 0 \right) & \geq \lim_{n \to \infty} \prod_{k=2}^n P(X_k = 0) \\ & = \lim_{n \to \infty} \prod_{k=2}^n \left( 1-\frac{1}{k^2} \right) \\ & \geq 1 - \lim_{n \to \infty} \sum_{k=2}^n \frac{1}{k^2} \\ & = 2 - \frac{\pi^2}{6} > 0 \end{align}

so the limiting sum

$$ \lim_{n \to \infty} \frac{1}{\sqrt{n}} \sum_{k=2}^n X_k $$

has an impulse at $0$ and is not normally distributed. (Having different distributions allows the probability of each successive random variable equaling $0$ to approach $1$, such that the probability that the partial sums never leave $0$ is strictly positive.)

Identical but not independent. Suppose $X_1$ is given by

$$ X_1 = \begin{cases} -1 & \text{with probability $1/2$} \\ 1 & \text{with probability $1/2$} \end{cases} $$

and $X_k, k \geq 2$ is given by

$$ X_k = \begin{cases} X_{k-1} & \text{if $k$ is a perfect square} \\ -X_{k-1} & \text{otherwise} \end{cases} $$

Then each $X_k$ has mean $0$ and variance $1$, but the limiting sum

$$ \lim_{n \to \infty} \frac{1}{\sqrt{n}} \sum_{k=1}^n X_k $$

has impulses at $-1$ and $1$ (probability $1/2$ each) and is not normally distributed. (Having dependent distributions allows us to force the partial sums into two distinctly different but deterministic directions.)

Answer 2

Lets say Prof Scatterbrain is trying to demonstrate to his large class of 100 students the truth of the central limit theorem.

This theorem states roughly that "If you have a population with mean μ and standard deviation σ and take sufficiently large random samples from the population, then the distribution of the sample means will be approximately normally distributed."

He gets his hands on the results on the 2023 Boston Marathon sorted by bib number going from 1 to 20,000. (Note: I downloaded the actual results for authenticity).

He hands his secretary these results and asks her to type them up. He wants her to take the first 200 finishing times and put them on a single page. Type the next 200 on another page and so on until he has 200 independent finishing times for each student.

The secretary gets bored after 2 pages and just photocopies them 50 times so that she has 100 sheets of paper which she gives to the professor the next morning. The professor hands them out to the class and asks that each student calculate the mean of the results they have and to bring their answer in the next day.

The next day the professor compiles a histogram of the means supplied by the students, which he expects will look like a normal distribution. He is expecting something like this (This is actually from the real results and looks roughly normally distributed):

But this is what he actually gets (Again, real, but only two different sets. It does not look normally distributed).

The Professor quickly ends the class amid raucous laughter.

And there you have an example of when the data is not IID (specifically not independent) and therefore the Central Limit Theorem does not apply.