What is the difference between weak and strong laws of large number?

Question

I've read the few posts on SE about weak vs strong law of large numbers, but I still can't quite differentiate the 2. Mathematically, it looks like the limit is applied to the probability, whereas in the weak law the limit is applied to the event.

WLLN: $$ \lim_{n \rightarrow \infty} P(|\bar{X}_n - \mu| > \epsilon) = 0 $$

SLLN: $$ P(\lim_{n \rightarrow \infty} |\bar{X}_n - \mu| = 0) = 1 $$

But I don't understand the significance of having the limit on the event vs. placing it on the probability. The WLLN, to me, in words says "probability that the absolute difference between the sample and population mean is greater than some arbitrary $\epsilon$ approaches zero as the sample size grows. The SLLN, to me, in words says "probability that the absolute difference between the sample and population mean being zero as the sample size grows approaches 1." These sound identical to me, just worded differently.

In addition, I also don't understand when one would be applicable over the other and am looking for an intuitive example.

Answer 1

Here's a good example, that relies on Borel-Cantelli. We will consider two sequences $A_1, A_2, A_3, ...$ and $B_1, B_2, B_3, ...$ of independent Bernoulli ($\{ 0, 1 \}$-valued) random variables. Suppose that

\begin{align*} P(A_n = 1) &= \frac{1}{n+1} \\ P(B_n = 1) &= \frac{1}{(n+1)^2} \end{align*}

Then $A_n \rightarrow 0$ in probability (the likelihood of $A_n > \epsilon$ goes to $0$ as $n \rightarrow \infty$), but almost surely, $A_n = 1$ infinitely often, by second Borel-Cantelli. So with probability $1$, $\lim_{n \rightarrow \infty} A_n$ does not exist. $A_n \rightarrow 0$ in the way that the r.v.'s in the weak law approach $0$: in probability, but not almost surely.

By contrast, because $\sum_{n=1}^\infty P(B_n = 1) = \sum_{n=1}^\infty \frac{1}{(n+1)^2} = \frac{\pi^2}{6} - 1$, $B_n = 0$ for all but finitely many $n$ with probability $1$, by Borel-Cantelli. So $\lim_{n \rightarrow \infty} B_n = 0$ almost surely, which is a much stronger statement than that $B_n \rightarrow 0$ in probability. $B_n \rightarrow 0$ in the way that the r.v.'s in the strong law approach $0$: almost surely.

So the SLLN is a much stronger statement, because the SLLN says the limit of the sequence of r.v.'s exists with probability $1$ and is equal to a certain number, while the WLLN just says the r.v's have higher and higher probability of being found near a certain number.

Edit: A nice description of the difference between these two modes of convergence can be given as follows. Let $X_1, X_2, X_3, ...$ be a sequence of random variables, and define the events $$E_n := \{|X_n - c| \geq \epsilon\}.$$ Then:

\begin{align*} X_n \rightarrow c \text{ in probability } &\leftrightarrow \lim_{n \rightarrow \infty} P(E_n) = 0 \text { for all } \epsilon > 0; \\ X_n \rightarrow c \text{ almost surely } &\leftrightarrow \lim_{n \rightarrow \infty} P\left( \bigcup_{k \geq n} E_k \right) = 0 \text { for all } \epsilon > 0. \ \end{align*}

Answer 2

In terms of gambling, the SLLN implies that for every gambler repeatedly playing a game, if you keep track of his average winnings, the average will get (and remain) arbitrarily close to the expected value. Even a game with slight negative expectation, the average winnings will, eventually, get and remain negative no matter how many additional games are played.

The WLLN allows for the possibility that, as you track the average winnings of a single gambler, the average winning can actually deviate from the expected value by some fixed value c and infinite number of times, although the times that this happens must happen more and more infrequently. In a case like this, it is conceivable that no matter how long a gambler has been playing a game with negative expected value, if he has sufficient wealth and patience, he would be able to climb out of a hole, eventually.

An example is to toss a fair coin until you get heads for the first time. Let $X$ denote the number of required tosses. Imagine you're paid $\frac{(-1)^X2^X}{X}$. Expected value does not exist, as the defining series converges conditionally (but not absolutely) to $-\ln 2$. So the WLLN holds but the SLLN does not. (Note the the value of the series converges to $-\ln 2$ but only in the given arrangement of terms. By rearranging the terms of the series, it can be made to converge to any real number one chooses: this is a property of conditionally convergent series that distinguishes from absolutely convergent series.)

More can be found here:

https://en.wikipedia.org/wiki/Law_of_large_numbers