Gambler's fallacy and the Law of large numbers

Question

Can someone explain me, how the Law of large numbers and the Gambler's Fallacy do not contradict.

The Gambler's Fallacy says, that there is no memory in randomness and any sequence of events has the same probability as any other sequence.

However, the Law of large numbers says, that given enough repetitions a certain event will likely happen.

To my understanding, these two kinda contradict each other because one says that you can not predict any random event but the other one says so (given enough repetitions of course).

For example imagine a series of coin tosses where the coin comes up heads a million times. The Gambler's fallacy says that the chance for the next toss to be tails is still 1/2. However the law of large numbers says, that since enough repetitions of tosses have come up heads, the next toss is more likely to be tails. (Which is definitely wrong?)

Answer 1

The keyword here for me is given.

...that given enough repetitions...

Your Gambler's Fallacy quote mentions that there is no memory in randomness, which is true, the events are independent. But the use of the word "given" introduces memory and future events become "dependant".

If I ask you:

What is the chance of flipping two heads (of a fair coin) in a row?

Your calculation would be:

$$ P(A \& B) = P(A) \times P(B) ={1\over 2} \times {1\over 2} = {1\over 4} $$

If I ask you:

Given that I just flipped heads, what is the chance of flipping two heads in a row?

Your calculation is:

$$ P(A | B) = { P(A \& B) \over P(A) } = {{1\over 2} \times {1\over 2} \over {1 \over 2}} = {1 \over 2} $$

---

_{Would be good if somebody could check my maths. It's been about 10 years since I did stats!}

Answer 2

Any sequence has the same probability as any other, but there are more sequences that are "balanced" than any other given proportion. For example, if I flip a coin 4 times then there are 6 ways to get 2 heads and 2 tails. There's only one way to get all heads though.

The Gambler's Fallacy compares individual sequences (for instance, the sequences HHHHH and HHHHT).

The LLN talks about groups of sequences - it says which groups your result is more likely to fall into.

Answer 3

This is an old post, but I am very surprised that no answer so far points out the exact issue, i.e. the exact mistake in the alleged contradiction (although @Deusovi's answer maybe does so, implicitly).

OP writes:

For example imagine a series of coin tosses where the coin comes up heads a million times. The [refutation of the] Gambler's fallacy says that the chance for the next toss to be tails is still 1/2. However the law of large numbers says, that since enough repetitions of tosses have come up heads, the next toss is more likely to be tails. (Which is definitely wrong?)

Indeed the sentence in boldface is wrong. That is not what the law of large numbers says! (It is, rather, exactly what the gambler's fallacy falsely believes.) What the law of large numbers says is that if you look at a very long sequences of coin tosses, and you assume that it is a fair coin, then on average, we expect half of them to be heads and half of them tails.

That means that if your first million tosses all gave heads, you have two options:

Option A) Still assume it is a fair coin, and keep tossing. Then the law of large numbers says that the next million coin tosses should roughly be half heads, half tails, so after that (but still counting your first million heads), starting with your rock solid faith in the coin being fair, you are now most reasonable in expecting to see a ratio of about $$\dfrac{1'500'000 \text{ heads}}{2'000'000 \text{ tosses}} \text{ versus } \dfrac{500'000 \text{ tails}}{2'000'000 \text{ tosses}},$$ i.e. a ratio of $75\% : 25\%$. If you toss ten million more times after your initial million heads, LLN tells you to expect $$\dfrac{6'000'000 \text{ heads}}{11'000'000 \text{ tosses}} \text{ versus } \dfrac{5'000'000 \text{ tails}}{11'000'000 \text{ tosses}},$$ i.e. a ratio of $\approx 54.5\% : 45.5\%$. If you toss a billion times more after your initial million heads, LLN tells you to expect $$\dfrac{501'000'000 \text{ heads}}{1'001'000'000 \text{ tosses}} \text{ versus } \dfrac{500'000'000 \text{ tails}}{1'001'000'000 \text{ tosses}},$$ i.e. a ratio of $\approx 50.05\% : 49.95\%$.

What we observe here is called regression to the mean, and here is a post where the top comment and answer say the same as what I want to say with the above. The Law of Large Numbers makes no statement about the probability of "the next" coin toss. If it makes a statement at all (by the way, one can doubt that meta-mathematically, cf. 1, 2, and note how I tried to word things carefully above in "based on assumption of fair coin ... most reasonable to expect ..."), then it makes a statement about the average of all tosses. Vaguely said, the fact that the ratio (here) goes to 50:50 is not due to the numerator on the right "catching up, with more tails now", but due to the denominator getting bigger, and the longer sequence eventually making the first million quite negligible.

Option B) Scrap the assumption that the coin is far. It is a very interesting question how to justify that and to come up with some "better" assumption on the (un)fairness of the coin. (The linked question basically asks the same as what you ask in this paragraph; note how most answers also start by saying we have the two options A and B that I'm talking about here; the answer by user307169 reiterates what I said in option A; the answer by Mark Fischler says how, if you go with option B, you are now faced with a new dichotomy between frequentist and Bayesian approaches; and the answer by Hayden gives what I think is the most appealing Bayesian approach on what to do next in option B.) However, then the Law of Large Numbers simply does not apply any more and thus makes no statement at all (in particular, no statement about what is likely in the next toss).

---

You see that either way, the law of large numbers does not say what it is claimed to say.

By the way, I personally would go with option B.