Question with Law of Large Numbers

Question

Let's say we have $$\frac{1}{n} \sum_{i=1}^n(X_i-\mu)^2$$ My teacher says that due to the Law of Large Numbers, this converges to $\mathbb{E}(X-\mu)^2$. I don't understand this. Why doesn't it converge to $\mathbb{E}(\overline{X}-\mu)^2$? What exactly is $X$?

Answer 1

In the Law of Large Numbers (LLN), suppose we have $X_1, \dots, X_n$ which are independent and identically distributed. Because $X_1, \dots, X_n$ are identically distributed, we can suppose, for example, that we have a random variable $X$ which follows the corresponding distribution.

Let $Y_i = (X_i - \mu)^2$ for $i = 1, \dots, n$ with $\mu = \mathbb{E}[X_i] = \mathbb{E}[X]$ (note: $X$ follows the same distribution as any of the $X_i$) for $i = 1, \dots, n$. The $Y_i$ are independent and identically distributed with mean equal to $\mathbb{E}[Y_i] = \mathbb{E}[(X_i - \mu)^2] = \mathbb{E}[(X - \mu)^2]$ for $i = 1, \dots, n$, and the LLN states that $$\bar{Y}=\dfrac{1}{n}\sum_{i=1}^{n}Y_i \to \mathbb{E}[Y_i]=\mathbb{E}[(X-\mu)^2]$$

as $n \to \infty$, and with some algebraic replacement, we have

$$\dfrac{1}{n}\sum_{i=1}^{n}(X_i - \mu)^2 \to \mathbb{E}[(X-\mu)^2]$$

as $n \to \infty$.