Probability of a single trial within binomial experiment vs. stand-alone bernoulli experiment

Question

When a flip a coin several times, each throw is independent from another. In other words, my coin does not know what came out previous time. So, each next flip the result is unpredictable and random. Now, suppose I flipped a fair coin three times and got each time a head (head-head-head). Intuitively, the head cannot come out “head” all the time, so I can expect the on fourth throw to have higher chances to get finally a tail. But each flip is an independent event - the coin does not know what came out last time.

How this paradox is resolved?

Many thanks!

Answer 1

Intuitively, the head cannot come out “head” all the time, so I can expect the on fourth throw to have higher chances to get finally a tail.

This marks the error.

Granted, there is no paradox - your error is known as the gambler's fallacy (or Monte Carlo fallacy). By assuming that tails is more likely, you assume that the trials are not independent even though they are. The fallacy is then that you feel the events are dependent, when they're not.

This was actually touched on in a recent Numberphile video somewhat: we think that streaks or heads of tails is nonrandom, i.e. would be a hint or sign of dependence or bias, when really they're not.

So there is no paradox, just bad intuition because of how human intuition tends to work.

Answer 2

If you flipped a fair coin three times and got heads each time, then on the next toss your probability of getting heads is $1/2$. The fact that you just flipped three heads in a row is irrelevant. There is no paradox.

Answer 3

"...so I can expect the on fourth throw to have higher chances to get finally a tail"

By independence there is no reason at all for expecting that, so there is no paradox.

Answer 4

If you already flipped three heads you already did the "hard part" of getting 4 heads in a row. Mathematically, if you look at the probability of four heads in a row with already three heads you should use conditional proababilities

$$P(\text{four heads in a row} | \text{I already got three heads in a row}) $$

(read as : "Given that I already got three heads in a row, what is the probability that I get four heads in a row") which is then, by independence of events, exactly $\frac{1}{2}$.

This might get a bit more intuitive if you look at more extreme cases. The probability of apocalypse in the next 5 minutes is (hopefully) very low. But if you see a nuclear warhead in the sky the probability of a nuclear apocalypse is high. The "hard part" (= unprobable events) already happened, so there is no paradox when $P(\text{apocalypse})$ is low but $P(\text{apocalypse}|\text{seeing a nuke})$ is high.

Answer 5

Intuitively, the head cannot come out “head” all the time, so I can expect the on fourth throw to have higher chances to get finally a tail

Not quite.

"In four tries, it's more probable to have 3 heads and 1 tail than to have 4 heads" => TRUE

"In four tries, the result HHHT is more probable than HHHH " => FALSE

This is a common confusion. See for example.

Answer 6

I find that numbers clear things up. For a fair coin,

$$\begin{align} &\Pr(\mathrm{HHH}) = \frac{1}{8}\\ &\Pr(\mathrm{HHHH}) = \frac{1}{16} \\ &\Pr(\mathrm{HHHT}) = \frac{1}{16} \\ \end{align}$$

That is, both the following are true:

• getting four heads in four coin tosses is somewhat rare (probability $1/16$: let's call it a “rarity of $16$”) before you start,

• but “half of the rarity” (i.e. a probability $1/8$ event) has already happened by the time you get three heads in a row, so that getting a further heads or tails are now equally likely again.

---

A point here is that all possible sequences of $\mathrm{H}$s and $\mathrm{T}$s of the same length have the same probability. That is, although it is true that:

$$\begin{align} \Pr(\text{$4$ Hs}) &= \frac{1}{16} \\ \Pr(\text{$3$ Hs, $1$ T}) &= \frac{4}{16} = \frac{2}{8} = \frac{1}{4} \\ \Pr(\text{$2$ Hs, $2$ Ts}) &= \frac{6}{16} = \frac{3}{8} \\ \Pr(\text{$1$ H, $3$ Ts}) &= \frac{4}{16} = \frac{2}{8} = \frac{1}{4} \\ \Pr(\text{$4$ Ts}) &= \frac{1}{16} \end{align}$$

it is still the case that $\mathrm{HHHH}$ and (say) $\mathrm{HTTH}$ are both equally likely (probability of $1/16$ each), and the “rarity” of “four heads in a row” comes from there being fewer ways for that to happen (or conversely, the higher probability of “two heads and two tails” comes from there being more ways for that to happen), rather than any individual sequence being less or more rare.

Richard Feynman in a lecture:

You know, the most amazing thing happened to me tonight. I was coming here, on the way to the lecture, and I came in through the parking lot. And you won’t believe what happened. I saw a car with the license plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance that I would see that particular one tonight? Amazing!