Convergence in Distribution of Sums of Random Variable

Question

Suppose I have $X_1,X_2,...,X_n$ random variables that are independent and identically distributed, from ANY distribution. Suppose that $E(X_i)=\mu$ and $V(X_i)=\sigma^2$.

Suppose I define the following random variable:

$$Y=\sum_{i=1}^nX_i$$

What is the limiting distribution of $Y$? That is, as $n$ goes to infinity, what distribution can $Y$ be approximated by?

My intuation tells me that $Y\rightarrow N(n\mu,n\sigma^2)$. In other words, say $200$ was a sufficiently large number for $n$. Then I could approximate $Y$ by a normal distribution with mean $200\mu$ and variance $200\sigma^2$. Is this true, and if so, how can you prove it? If not, what is the limiting distribution of $Y$?

Answer 1

Any statement that says $\lim_{n\to\infty}(\cdots\cdots) = (\text{something depending on $n$})$ is wrong if taken literally, and usually wrong if taken any other way.

The distribution $N(n\mu,n\sigma^2)$ depends on $n$ and does not approach a limit as $n$ grows.

However, the distribution of $$ \frac{Y-n\mu}{\sigma\sqrt n} \tag 1 $$ does approach a limit as $n$ grows (unless $\sigma=+\infty,$ as happens in some cases). That limit is $N(0,1).$

This may be understood as meaning that the c.d.f. of $(1)$ converges pointwise to the c.d.f. of $N(0,1).$ If the limit were a distribution that concentrates positive probability at some points, it would be understood as meaning that the c.d.f. converges pointwise except at points where the limiting distribution assigns positive probability.

Answer 2

The (modified, in the sense you will see!) result you are after follows directly from a class of convergence results called central limit theorems (of probability theory). There are several versions of central limit theorems, according majorly as the dependence and the distribution heterogeneity conditions vary. In the present case of your interest, we are concerned with the prototype of central limit theorem, dealing with a sequence of sums of independent identically distributed random variables. To prove it requires lots of prerequisites that I suspect you have not yet been exposed to; so let me present the idea of a typical proof instead. The typical proof scheme is to utilize the fact that weak convergence of distribution functions is the pointwise convergence of the corresponding characteristic functions. Then it can be shown by using the independence assumption and a second-order Taylor expansion that the sequence of standardized sums of random variables converges in distribution to a standard normal random variable. It turns out that the weak convergence of the distribution functions of the standardized sums of random variables to the standard normal distribution function is also uniform (even in the case where the random variables involved are independent but nonindentical, as long as the so-called Lindeberg condition is satisfied.). So, in fact, if $F_{Y_{n}}$ is the distribution function of $Y_{n} := \sum_{1}^{n}X_{i}$ and if $\Phi$ is that of a standard normal random variable, then we have $F_{Y_{n}}(y) - \Phi\big(\frac{y - n\mu}{n\sigma^{2}}) \to 0$ as $n \to \infty$ for all $y$. Then, precisely speaking, we say that $Y_{n}$ is distributed asymptotically as $N(n\mu, n\sigma^{2})$. I guess this is what you are after.

To sum: we say that $(Y_{n}-EY_{n})/\sqrt{\text{var}(Y_{n})}$ converges in distribution to $N(0,1)$ or that $N(0,1)$ is the limiting distribution of the sequence of the standardized $Y_{n}$, but that $Y_{n} \sim_{A} N(EY_{n}, \text{var}(Y_{n})$), reading $Y_{n}$ is asymptotically normally distributed with mean $EY_{n}$ and variance $\text{var}Y_{n}$.