Does a Gaussian random walk lead to a Gaussian distribution in the limit, even when the initial state is non-Gaussian?

Question

Suppose we have an initial random variable, which is not Gaussian, but has mean $0$, std $1$. Now we add $N$ unit Gaussian variables to this initial random variable, and then renormalize to mean $0$, std $1$. Call the resulting random variable $Z$.

In the limit as $N$ goes to infinity, is $Z$ a unit Gaussian distribution?

Answer 1

Let $X$ and $Y$ be random variables. A few facts:

• $\operatorname{E}[X + Y] = \operatorname{E}[X] + \operatorname{E}[Y]$
• If $X$ and $Y$ are independent, then $\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y)$.
• If $X$ and $Y$ are independent and Gaussian, then $X$ + $Y$ is also Gaussian.

So adding $N$ unit Gaussian variables is the same as adding one Gaussian variable with mean 0 and variance $N$. Let's call this variable $G_N$.

Now let $X$ be our initial random variable. After adding the Gaussian we have:

• $\operatorname{E}[X + G_N] = 0$
• $\operatorname{Var}(X + G_N) = 1 + N$

Then we can do: $$ \begin{array}{} Z &=& \lim_{N \to \infty} \frac{X + G_N}{\sqrt{1 + N}} \\ &=& \lim_{N \to \infty} \frac{X}{\sqrt{1 + N}} + \frac{G_N}{\sqrt{1 + N}} \\ &=& \lim_{N \to \infty} \frac{X}{\sqrt{1 + N}} + G_\frac{N}{1+N} \\ &=& G_1 \end{array} $$ Because $\lim_{N \to \infty} \frac{1}{\sqrt{1 + N}} = 0$ and $\lim_{N \to \infty} \frac{N}{1 + N} = 1$.