In this lecture, at about the $37$ minute mark, the professor explains how the binomial distribution, under certain circumstances, transforms into the Poisson distribution, then how as the mean value of the Poisson distr. increases, the devation from the mean behaves like a Gaussian. I'm having trouble with calculating this.
The pmf of the Poisson distr. is
$$p(n)=\frac{e^{-\lambda}\lambda^n}{n!}$$
If we define $x=\lambda-n$ to be the deviation from the mean and substitute into the pmf, we get
$$p(x)=\frac{e^{-\lambda}\lambda^{\lambda-x}}{(\lambda -x)!}$$
Using the Stirling formula, this becomes approximately
$$p(x) \approx \frac{e^{-\lambda}\lambda^{\lambda-x}}{(\lambda -x)^{\lambda -x} e^{-(\lambda -x)} \sqrt{2\pi (\lambda -x)} } = \frac{e^{-x}}{\sqrt{2\pi}} \frac{\lambda^{\lambda-x}}{(\lambda -x)^{\lambda -x}} (\lambda -x)^{-\frac{1}{2}}$$
$$=\frac{e^{-x}}{\sqrt{2\pi}} (\lambda -x)^{-\frac{1}{2}} (1-\frac{1}{\lambda /x})^{x-\lambda} = \frac{e^{-x}}{\sqrt{2\pi}} (\lambda -x)^{-\frac{1}{2}} ((1-\frac{1}{\lambda /x})^{\lambda /x})^{\frac{x^2-\lambda x}{\lambda}}$$
Now the rightmost term tends to $$e^{-\frac{x^2}{\lambda}}e^x$$
as $\lambda \to \infty$, which cancels the $e^{-x}$ in front. So this would be looking fairly Gaussian if not for the uncomfortable factor of $(\lambda -x)^{-\frac{1}{2}}$ which tends to zero, killing the whole thing!
Where am I making a mistake? Clearly, in the Stirling approximation, the square root factor isn't too relevant (order-wise), and if it's omitted then the problem goes away (or does it? the result isn't normalised!)... but surely this can be fixed?
It is ok, just notice that the analysis is done with $x$ is fixed so for large $\lambda$ we have $$ (\lambda - x ) \sim \lambda $$ And remember that the variance of a Poisson random variable is $\lambda$. So the resulting density is (almost) normal with mean zero and variance $\lambda$ which makes sense.
A more rigourous analysis can be made using the weak convergence machinery. If we consider $$Y_\lambda = \frac{X - \lambda}{\sqrt{\lambda}}$$ then its characteristic function is $$\phi_{Y_\lambda}(t) = \exp(-\sqrt{\lambda}it + \lambda(e^{\frac{it}{\sqrt{\lambda}}}-1))$$ For large $\lambda$ $$e^{\frac{it}{\sqrt{\lambda}}}-1 = \frac{it}{\sqrt{\lambda}} - \frac{t^2}{2\lambda} + o(\lambda^{-1})$$ And so in the limit $$\lim_{\lambda \to \infty} \phi_{Y_\lambda}(t) = \exp(-t^2/2)$$ which is the characteristic function of a normal$(0,1)$. So for large $\lambda$, $Y_\lambda$ is close to a normal distribution or equivalently $X$ is normally distributed with mean $\lambda$ and variance $\lambda$