approximation to standard normal distribution

Question

I have a sequence of independent, but non-identically distributed Bernoulli random variables $X_i$'s taking on value $1$ with probability $p_{i1}\cdot p_{i2}$, where $i=1,\ldots,n$. Let $X=\sum_{i=1}^n X_i$. Using Lyapunov central limit theorem, I have approximated the probability $\Pr[X \geq k]$ by standard normal distribution. However, for some values of $k$ the approximated value is not close to the exact value. I wanted to ask that why this happens? Thanks in advance for your help.

Answer 1

Summary

Notation: Let $F_n$ be the cumulative distribution function for $X = \sum_{I=1}^n X_i$, and let $F$ be the cumulative distribution function $Z$ where $Z$ follows a standard normal distribution.

The result that you have only establishes that for any fixed $k$, $P[Z \geq k]$ will be an increasingly good approximation to $P[X \geq k]$ as $n \rightarrow \infty$.

Details

You have convergence in distribution (by the Lyapunov central limit theorem): for any $x \in \mathbb{R}$, $F_n(x) \rightarrow F(x)$ as $n \rightarrow \infty$. (This holds for any real value $x$ since $F$ is continuous on the real numbers $\mathbb{R}$.)

But (I think that) what you're seeing is that for any given $n$, the closeness of $F_n(x)$ to $F(x)$ can vary with $x$. Hence in particular, the closeness of $F_n(k)$ to $F(k)$ can vary as $k$ increases. Therefore since $P[X \geq k] = 1 - F_n(k)$ and $P[Z \geq k] = 1 - F(k)$, the closeness of $P[Z \geq k]$ to $P[X \geq k]$ can also vary as $k$ increases.

In other words, the convergence in distribution is pointwise convergence and not uniform convergence. That is, in establishing that for any $x \in \mathbb{R}$, $F_n(x) \rightarrow F(x)$ as $n \rightarrow \infty$, the following is true: $$ \text{pointwise convergence:}\ \text{for any}\ x \in \mathbb{R}\ \text{and for any}\ \epsilon > 0, \text{there exists}\ N > 0\ \text{such that if}\ n \geq N \ \text{then}\ |F_n(x) - F(x) |< \epsilon $$ The following is not necessarily true: $$ \text{uniform convergence:}\ \text{for any}\ \epsilon > 0, \text{there exists}\ N > 0\ \text{such that for any}\ x \in \mathbb{R}, \text{if}\ n \geq N \ \text{then}\ |F_n(x) - F(x) |< \epsilon $$