Applications of the CLT for the Sample Mean

Question

Related to Confusion of central limit theory, and the fact that I just finished my first course in (Master's-level) graduate-level probability, which relates to this material.

The Central Limit Theorem states that if you have an iid sample $X_1, \dots, X_n$ with mean $\mu$ and variance $\sigma^2<\infty$, denoting $\bar{X}_n = \dfrac{1}{n}\sum_{i=1}^{n}X_i$, we have

$$\sqrt{n}(\bar{X}_n - \mu) \overset{d}{\to}\mathcal{N}(0, \sigma^2)$$ as $n \to \infty$. By Slutsky's theorem, since $\sigma = \sqrt{\sigma^2}$ is constant (let's assume in addition $\sigma \neq 0$), obviously $\sigma \overset{p}{\to}\sigma$, hence $$\dfrac{\bar{X}_n - \mu}{\sigma/\sqrt{n}}\overset{d}{\to}\dfrac{1}{\sigma}\cdot\mathcal{N}(0, \sigma^2) = \mathcal{N}(0, 1)$$ as $n \to \infty$. This justifies (to me) what is commonly what is done in intro stats classes: basically, if $n \geq 30$, if you want to, say, find $$\mathbb{P}(a \leq \bar{X}_n \leq b)$$ where $a$ and $b$ are usually finite, the idea is that when you standardize it as follows: $$\mathbb{P}(a \leq \bar{X}_n \leq b)\approx\mathbb{P}\left(\dfrac{a-\mu}{\sigma/\sqrt{n}} \leq \dfrac{\bar{X}_n - \mu}{\sigma/\sqrt{n}} \leq \dfrac{b-\mu}{\sigma/\sqrt{n}}\right)$$ you can approximate $$\dfrac{\bar{X}_n - \mu}{\sigma/\sqrt{n}}$$ to be a $\mathcal{N}(0, 1)$ random variable if $n$ is "large enough;" the standard usually being $30$.

Now the question I have: this implies, then, that we can't necessarily assume $$\bar{X}_n\overset{d}{\to}\mathcal{N}\left(\mu, \dfrac{\sigma^2}{n}\right)$$ as $n \to \infty$. Is it correct that this isn't true?

For one thing, this makes no sense, as $n$ is still showing up in the normal distribution (in the variance) as $n \to \infty$. Obviously, Slutsky's theorem will not work (as far as I can tell).

But this seems to be contradicted by a select few websites:

https://onlinecourses.science.psu.edu/stat800/node/36

http://onlinestatbook.com/2/sampling_distributions/samp_dist_mean.html

What am I missing?

Edit: The reason why I believe Slutsky's Theorem will not work is as follows: denote $Y_n = \dfrac{\bar{X}_n - \mu}{\sigma/\sqrt{n}}$ and let $F_{Z}$ denote the CDF of $Z \sim \mathcal{N}(0, 1)$. Then for all $y \in \mathbb{R}$ for which $F_{Y_n}$ is continuous (notice, particularly, that $y$ is a constant), $$\lim_{n \to \infty}F_{Y_n}(y) = F_{Z}(y)$$ but $$F_{\bar{X}_n}(x) = \mathbb{P}(\bar{X}_n \leq x) = \mathbb{P}\left(\dfrac{\sigma Y_n}{\sqrt{n}}+\mu \leq x\right) = \mathbb{P}\left(Y_n\leq\dfrac{x-\mu}{\sigma/\sqrt{n}}\right) = F_{Y_n}\left(\dfrac{x-\mu}{\sigma/\sqrt{n}}\right)\text{.}$$ Unfortunately, the argument $\dfrac{x-\mu}{\sigma/\sqrt{n}}$ is dependent on $n$, so Slutsky's Theorem won't help.

Answer 1

$$\bar{X}_n\overset{d}{\to}\mathcal{N}\left(\mu, \dfrac{\sigma^2}{n}\right)\tag{1}$$ as $n \to \infty$.

You don't need any deep mathematics to see why (1) is not true. Because the notations in (1) does not make sense at all.

Think about the following example. Note that you have the following limit $$ \lim_{n\to\infty}\frac{1}{1+n}=0. $$

On the other hand, for $n$ large enough, one can approximate $\dfrac{1}{1+n}$ by $\dfrac{1}{n}$ because $\lim_{n\to\infty}\biggr(\frac{1}{1+n}-\frac{1}{n}\biggr)=0$. However, one cannot say $$ \frac{1}{n+1}\to\frac{1}{n} \quad \text{as } n\to\infty.\tag{2}$$

Answer 2

Nice question! If the problem is that the variance vanishes as $n \to \infty$, here's an intuitive explanation. Let's see it from this view point: Why does variance exists at all? Because we are uncertain about the random variable $X$. The variance of $\overline{X_n}$ is a measure of uncertainty that we have about $\overline{X_n}$, which obviously is generated from our uncertainty about $X$. From a statistician's viewpoint, the more data you can obtain, the less is the uncertainty. Here as $n$ increases, we obtain more data about $X$, and hence, as expected, the variance of $\overline{X_n}$ decreases. Now if $n$ approaches $\infty$ (but not equal to $\infty$, of course), the variance can be made arbitrarily small (but still positive!). In other words, the statistic $\overline{X_n}$ converges to $\mu$ with arbitrarily small uncertainty, i.e. for a pre-fixed $\delta > 0$, the quantity $P(|\overline{X_n}-\mu|>\delta)$ can be made arbitrarily small by choosing $n$ large enough. So the uncertainty, while cannot be fully conquered, can be controlled to arbitrary precision by obtaining more and more data.

NOTE THAT it might seem that the variance goes to zero and the distribution becomes degenerate. But that's NOT happening here! The graph of the probability density function may show a straight vertical line (unit mass) on the point $\mu$, but that's only in bare eyes. Instead, if you can look at it by changing the scale accordingly (like using a magnifying glass), you'll see that it's a normal bell-curve, which only look like a straight vertical line in our bare eyes.

Answer 3

Your notation is sloppy and confused me quite a while. If we put this way: $$ F_{\sqrt{n} (\overline{X}_n -\mu)} \overset{d}{\to} \mathcal{N}(0,\sigma^2) $$ implies $$ F_{\sqrt{n} (\overline{X}_n -\mu)} -\mathcal{N}(0,\sigma^2) \overset{d}{\to} 0 $$ implies (explained just below) $$ F_{\overline{X}_n -\mu} -\mathcal{N}(0,\sigma^2/n) \overset{d}{\to} 0 \quad ...[\spadesuit] $$ And then $$ F_{\overline{X}_n} -\mathcal{N}(\mu,\sigma^2/n) \overset{d}{\to} 0 $$ I.e., averaged $\overline{X}_n$ resembles, pointwise, the Gaussian with $\mu$ and $\sigma^2/n$. May be this just better captures what laymen mean by "oh, you see, this becomes close to Gaussian!"

The $[\spadesuit]$ follows by no means of difficult probability theory. The claim is pointwise: when 1st term approximates 2nd term for some $x$, then same is true when the prob. mass is scaled correspondingly, so we are comparing the very same density times $\sqrt{n}$ only now at $x/\sqrt{n}$ (more exactly $x$'s for which r.h.s. is conti., but in this short section everything at r.h.s. is conti.). The following (last) step is even easier, since we are just comparing the same thing with a shift $\mu$ w.r.t. $x$. I leave to the reader how it is said with lovely $\epsilon$ and $\delta$.