The Poissonian PDF tends to a Gaussian for $\lambda\rightarrow\infty$

Question

I found an argument for the title in this question, using the Stirling approximation and defining $x=\lambda-k$, $$ \frac{\lambda^k}{k!}e^{-\lambda}=\frac{\lambda^{\lambda-x}e^{-\lambda}}{(\lambda-x)!} \approx \frac{e^{-\lambda}\lambda^{\lambda-x}}{(\lambda -x)^{\lambda -x} e^{-(\lambda -x)} \sqrt{2\pi (\lambda -x)} } =\\ = \frac{e^{-x}}{\sqrt{2\pi}} \frac{\lambda^{\lambda-x}}{(\lambda -x)^{\lambda -x}} (\lambda -x)^{-\frac{1}{2}}=\\ = \frac{e^{-x}}{\sqrt{2\pi}} (\lambda -x)^{-\frac{1}{2}} \left(\left(1-\frac{1}{\lambda /x}\right)^{\lambda /x}\right)^{\frac{x^2-\lambda x}{\lambda}}\approx\frac{1}{\sqrt{2\pi\lambda}}e^{-\frac{(k-\lambda)^2}{\lambda}}$$ as $\lambda\rightarrow\infty$. The problem is that the final pdf at which they arrive isn't normalized in reason of the missing factor of 2 in the denominator inside the exponential. I can't see where that factor is lost, any help?

Answer 1

I've identified a couple of things that went wrong. First of all if you write things in terms of $x$ then the PDF goes to 0 anyway. This allows you to do manipulations that are technically correct but not helpful. Note that all expressions are the same in the limit $\lambda\to\infty$ simply because all of them go to 0.

To make things rigorous it's better to write things in terms of $y = x/\sqrt{\lambda}$, since $y$ should have a variance of 1 we can keep it fixed and actually take the limit without having to worry about the PDF vanishing. Note that if we perform the coordinate transformation $x \to \sqrt\lambda y$ then we should also multiply by $\sqrt{\lambda}$ to keep everything normalized.

Now if we do this you'll find that the $e^{-x}$ becomes $e^{-\sqrt\lambda y} $ which goes to 0, and this is multiplied by a term going to $\infty$. Clearly it's not good to take those limits separately and multiply the result, which you implicitly did in your derivation. If you fix this you'll also find that the limit $\lim_{x\to\infty} (1+1/x)^x = e$, or equivalently $\lim_{x\to\infty} \exp(x\log(1+1/x)) = e$, won't converge quickly enough for our purposes, which will result in an extra term that fixes the normalisation.

Now for readability I'll also write $\sigma=\sqrt\lambda$. This results in:

$$ \begin{align} \lim_{\lambda\to\infty}& \frac{e^{-\sqrt\lambda y}}{\sqrt{2\pi}} \frac{\sqrt\lambda}{\sqrt{\lambda - \sqrt{\lambda}y}}\left(\frac{\lambda}{\lambda - \sqrt\lambda y} \right)^{\lambda - \sqrt\lambda y}\\ &= \lim_{\sigma\to\infty} \frac{e^{-\sigma y}}{\sqrt{2\pi}} \sqrt\frac{\sigma^2}{\sigma^2 - \sigma y} \left(\frac{\sigma}{\sigma - y} \right)^{\sigma^2 - \sigma y} \\ &= \lim_{\sigma\to\infty} \frac{e^{-\sigma y}}{\sqrt{2\pi}} \left(\frac{\sigma}{\sigma - y} \right)^{\sigma^2 - \sigma y} \\ &= \lim_{\sigma\to\infty} \frac{1}{\sqrt{2\pi}} \exp\bigl(-\sigma y + (\sigma^2 - \sigma y) (\log(\sigma) - \log(\sigma - y))\bigr)\\ &= \lim_{\sigma\to\infty} \frac{1}{\sqrt{2\pi}} \exp\left(-\sigma y + (\sigma^2 - \sigma y) \left(\log(\sigma) - \log(\sigma) + \frac{y}{\sigma} + \frac12 \frac{y^2}{\sigma^2} + o\left(\frac{y^2}{\sigma^2}\right)\right)\right)\\ &= \lim_{\sigma\to\infty} \frac{1}{\sqrt{2\pi}} \exp\left(-\sigma y + (\sigma^2 - \sigma y) \left(\frac{y}{\sigma} + \frac12 \frac{y^2}{\sigma^2} + o\left(\frac{y^2}{\sigma^2}\right)\right)\right)\\ &= \lim_{\sigma\to\infty} \frac{1}{\sqrt{2\pi}} \exp\left(-y^2 + \frac12 (\sigma^2 - \sigma y) \frac{y^2}{\sigma^2} + (\sigma^2 - \sigma y) o\left(\frac{y^2}{\sigma^2}\right)\right) \\ &= \lim_{\sigma\to\infty} \frac{1}{\sqrt{2\pi}} \exp\left(-y^2 + \frac12 \sigma^2 \frac{y^2}{\sigma^2}\right)\\ &= \frac{1}{\sqrt{2\pi}} \exp\left(-\frac12 y^2\right) \end{align} $$

Note that if we'd approximated $\log(\sigma - y)$ to first order we'd have gotten:

$$ \lim_{\sigma\to\infty} -\sigma y + (\sigma^2 - \sigma y) \left(\frac{y}{\sigma} + o\left(\frac{y}{\sigma}\right)\right) = \lim_{\sigma\to\infty} -y^2 + (\sigma^2 - \sigma y) o\left(\frac{y}{\sigma}\right) = \lim_{\sigma\to\infty} -y^2 + \sigma^2 o\left(\frac{y}{\sigma}\right) $$

But here the term $ \sigma^2 o\left(\frac{y}{\sigma}\right)$ doensn't go to 0 quickly enough, so if you incorrectly assume it vanishes then you get the wrong answer, resulting in the missing factor $\frac12$.