Convergence of average of i.i.d. Bernoulli to Normal will be slower for p closer to 0 or 1

Question

this is true. If you fit a normal to the data you see, if you have small samples, the normal is spread out in shape. So the area cut out by the $x=0$ line or $x=1$ line is larger than the case when $p=0.5$. Any other intuitive explanation to back this claim? How do people go about proving this statement? Thanks.

Answer 1

A Bernoulli$(p)$ distribution has $\mu=p$ , $\sigma^2=p(1-p)$

By the CLT , the CDF of the variable $Z=\sum (X_i-p)/\sqrt{n p (1-p)}$ will tend to a standard normal distribution $\Phi(z)$.

The Berry–Esseen theorem gives a bound:

$$ F_Z(z)-\Phi(z)\le \frac{C\rho}{\sigma^3} \sqrt{n}$$

where $C\approx 0.4$ and $\rho=E(|(X-\mu)^3|)$. In our case $\rho=\left( 1-p\right) \,{{p}^{3}}+{{\left( 1-p\right) }^{3}}p$

The factor $\rho/\sigma^3$ has a mininum for $p=1/2$, and grows without bound at the extremes. Hence, it's to be expected (though this is not a proof) that the convergence is slower when $p\to 0$ or $p\to 1$.

You could also revisit several proofs of convergence and examine the limiting arguments when $p\to 0$. For example, this answer reveals that negligible terms at the limit ($n \to \infty$) actually depend on $\sqrt{npq}$ going to infinite. Hence, if some $n_1$ is "big enough" for, say $p=0.01$, then for $p=0.001$ we'll need a new $n$ ten times bigger.