Almost Sure Convergence for Sample Mean of Bernoullis

Question

Let {$B_i$} be a sequence of Bernoulli($\mu$) variables and $X_n$ its sample mean $X_n=\frac1n\sum_i^nBi$. Because of the Strong Law of Large Numbers, we know that $X_n$ converges almost surely to $\mu$.

I am not sure if I completely understand what this implies. For a given n, there are sets of {$B_i$} for which $X_n$ doesn't converge to $\mu$ (for example, [1,1,1,1,1,1,1,1,1...]) and which doesn't have probability cero. If $n \rightarrow \infty$, do this non converging sets have probability cero?

Then, almost sure convergence states that for any $\epsilon > 0$, there is a $n_0$ such that if $n > n_0$ then $|X_n-\mu| < \epsilon$. But right after any $n_0$ I could have a secuence of many 1s such that $X_n$ could eventually grow past $\mu+\epsilon$. Does this have probability cero of ocurrence? How do I found a $n_0$ for any given $\epsilon$?

Answer 1

After further studying the subject I have reached an answer.

What I have stated in my question does not always applies for discrete sequences of variables. The hypothesis needed for the Strong Law of Big Numbers is different for discrete and continuous sequences. Therefore, the sample mean of a sequence of Bernoullis doesn't converge almost surely, although it does in probability.

Answer 2

The proper definition of "sequence $B_i$ summed from 1 to $n$ converges in probability to $\mu$ as $n \rightarrow \infty$" is:

For any specified value $\epsilon > 0$ and probability $\alpha > 0$, there exists an $n_0$ such that for all $n > n_0$ $$ \mbox{Prob } \left( \left| \frac{1}{n}\sum_1^n B_i - \mu \right| > \epsilon \right) < \alpha $$ For $B_i$ independent identical Bernoulli distributed with mean $\mu$, this is reasonably easy to show; see for example Feller's Theory of Probability Volume 1.

The cases you illustrate contribute terms to the probability of violating the mean by at least $\epsilon$, which however are vanishingly small for sufficiently large $n_0$. In fact, it is possible to determine an adequate value of $n_0$ as a function of $\epsilon$ and $\alpha$.