Why is the Gambler's Fallacy a fallacy? [Solved]

Question

The empirical probability distribution of the coin tossing bet is binomial distribution. For a fixed length of sequential events, it shows that the more balance the number of Heads strikes between the number of Tails, the more probable it will occur.

With the gambler's fallacy, I will bet the coin tossing like this:

Given the coin has been tossed $6H3T$ without knowing the order, and I have $100$ USD gambit to distribute to H or T for the next random event.

$P(7H3T)=C(10,3)*(0.5)^{10}=0.1171875.$

$P(6H4T)=C(10,4)*(0.5)^{10}=0.205078125.$

Hence I will distribute $100* (0.117)/(0.117+0.205)=36$ USD to H, and $64$ USD to T.

Why is this gambler's analysis considered a fallacy, where sequential binomial distribution is considered?

---

Edit: Let me modify the bet. Given the 10 coins have been tossed at the same time. 9 random coins are revealed as $6H3T$, and I have $100$ USD gambit to distribute to H or T for guessing the unrevealed coin.

---

Edit: I recognise the independence of the respective events, but suggest and am confused as to whether there is dependence introduced by the binomial distribution - the density of each binary follows everywhere.

---

Edit: Toss 3 coins at the same time. Hide one fairly. The reveal (abbreviated as RV) is 2T.

Possible outcomes including the unrevealed (abbreviated as POIU): 1H2T, 3T.

The probability that POIU is 1H2T and RV is 2T =(3/8)(1/3)=1/8.

The probability that POIU is 3T and RV is 2T =(1/8)(1)=1/8.

P(POIU=3T|RV=2T)=P(POIU=3T ∩ RV=2T)/P(RV=2T)=(1/8)/(1/4)=1/2.

P(POIU=1H2T|RV=2T)=P(POIU=1H2T ∩ RV=2T)/P(RV=2T)=(1/8)/(1/4)=1/2.

The fallacy I made is assumming every POIU under fair hiding must lead to RV.

The confusion is solved. Thanks for all the answers.

Answer 1

Given the coin is tossed 6H3T without knowing the sequence

You say this, but then you don't use the math for conditional probability:

$$P(7H3T|6H3T) = P(\textrm{7H3T with the last flip H}) / P(6H3T) = \frac{C(9,3)2^{-10}}{C(9,3)2^{-9}} = \frac{1}{2}.$$

Or to drive the point home a bit more: given that the coin has been tossed 6H3T, what is the probability that after the next flip the totals will be 5H5T? Clearly 0, right? You can't change the past? And not $$P(5H5T) = C(10,5)2^{-10} \approx 0.25?$$

Of course all of this is just to say that the coin flips are independent: the fact that the coin has flipped 6H3T in the past (assuming you also know that the coin is fair) yields no information about the next flip.

Answer 2

Information entropy quantifies the uncertainty associated with a random variable. For a fair coin toss, the random variable $X$ representing the outcomes has an associated information entropy

$$ H(X) = -\sum p(x_i) \log_2 p(x_i), $$

where $x_i$ represents each possible outcome (heads or tails), and $p(x_i) = 0.5$ for a fair coin. This yields

$$H(X) = 1 \, \text{bit}. $$

Every toss of the coin is an independent event with an associated entropy of 1 bit. This independence is key; the entropy, and hence the uncertainty, remains constant for each toss. There's no reduction in uncertainty, and no "memory" of past events that influences future outcomes.

The binomial distribution can describe the distribution of outcomes over a fixed number of trials. The most likely distribution is one where the outcomes are balanced, as it has the maximum entropy and hence maximum uncertainty. However, it's crucial to understand that this doesn't imply a dependency between individual trials.

Each specific sequence of outcomes (like HHTHT, HTTHH, etc.) has an equal probability of occurrence in a fair coin toss scenario. The gambler's fallacy arises from the erroneous belief that an "imbalance" in outcomes (e.g., a run of heads) must be "corrected" by future tosses to achieve the more probable balanced state, which is a misunderstanding of the nature of independent events. Each toss remains an independent event, governed by the same level of uncertainty, quantified by the information entropy $H(X) = 1 $ bit. If you like this answer I suggest you try and read Claude Shannon's original paper on information entropy, it's easily found online.

Answer 3

Flip three coins and reveal the results one-at-a-time.

There are only 8 total possibilities:

HHH, HHT, HTH, HTT, THH, THT, TTH, TTT

Each of those orderings is equally likely to happen 1/8 of the time.

---

If we imagine that we've already rolled and revealed the first two coins:

HH?

What are the odds that HHH was rolled?

---

Gambler's Fallacy

If we think to ourselves a bit, we might notice that there is only 1 way (HHH) to get three heads whereas there are 3 ways (HHT, HTH, THH) to get two heads and one tails.

Knowing that, we might decide "Three heads is three times rarer than two heads and one tails." and decide that the final result is only 25% likely to be HHH and 75% otherwise.

But we goofed!

How? Because the odds of HTH and THH happening have nothing to do with the probability of what HH? can be after the first two coins have already been revealed.

The 25/75 odds split imagines that HHH is one of four possibilities (HHH,HHT,HTH,THH) that are each equally likely to happen... but after the first two coins have been revealed, the HTH and THH possibilities are ruled out entirely; those possibilities have 0% probability now. The only two possible results now are either HHT or HHH.

So the odds that HHH was rolled are just as good as the odds that HHT was rolled. The last coin is either H or T with equal probability! The odds on the last coin are 50/50, independently of how rare or common the particular sequence of previous revealed coins was!

Answer 4

The Gambler's Fallacy is called that because people make mistakes.

Let's take your statement:

$10$ coins have been tossed at the same time. $9$ random coins are revealed as $6H3T$.

Your calculation of probabilities should be, but are not, conditioned on the $6H3T$ observation of the $9$ coins.

We will call

• event $A_{7H3T}$ that the $10$ coins are revealed as $7H4T$
• event $A_{6H4T}$ that the $10$ coins are revealed as $6H4T$
• event $B_{6H3T}$ that $9$ coins chosen from the $10$ are revealed as $6H3T$

You have found that before anything is observed

• $P(A_{7H3T})=C(10,3)\times 0.5^{10} =120\times 0.5^{10}$
• $P(A_{6H4T})=C(10,4)\times 0.5^{10} =210\times 0.5^{10}$

Using the hypergeometric distribution, we can find the conditional probabilities of observing $B_{6H3T}$:

• $P(B_{6H3T} \mid A_{7H3T})=\frac{C(7,6)\times C(3,3)}{C(10,9)}=\frac{7}{10}$
• $P(B_{6H3T} \mid A_{6H4T})=\frac{C(6,6)\times C(4,3)}{C(10,9)}=\frac{4}{10}$

And combining these gives the joint and marginal probabilities

• $P(B_{6H3T} \cap A_{7H3T})=P(B_{6H3T} \mid A_{7H3T})\times P(A_{7H3T}) =\frac{7}{10} \times 120\times 0.5^{10} = 84\times 0.5^{10}$
• $P(B_{6H3T} \cap A_{6H4T})=P(B_{6H3T} \mid A_{6H4T})\times P(A_{6H4T}) =\frac{4}{10} \times 210\times 0.5^{10} = 84\times 0.5^{10}$
• $P(B_{6H3T})=P(B_{6H3T} \cap A_{7H3T})+P(B_{6H3T} \cap A_{6H4T}) =168\times 0.5^{10}$

Which means the conditional probabilities we are really interested in are:

• $P(A_{7H3T} \mid B_{6H3T}) = \frac{P(B_{6H3T} \cap A_{7H3T})}{P(B_{6H3T})} =\frac{84\times 0.5^{10}}{168\times 0.5^{10}} =\frac12$
• $P(A_{6H4T} \mid B_{6H3T}) = \frac{P(B_{6H3T} \cap A_{6H4T})}{P(B_{6H3T})} =\frac{84\times 0.5^{10}}{168\times 0.5^{10}} =\frac12$

i.e. the unrevealed coin is equally likely to be heads or tails.

Answer 5

There is an experiment we can perform that produces probabilities like the ones you wrote in your question.

We start by flipping $10$ coins all at once. If there are not at least $6$ heads and at least $3$ tails in this set of flips, we gather up all the coins and flip them all again.

Eventually, with probability $1$, we'll flip the ten coins and have at least $6$ heads and at least $3$ tails. When that happens, we stop flipping the coins.

At that point we will either have $7$ heads and $3$ tails ($7H3T$) or $6$ heads and $4$ tails ($6H4T$). What is the probability of each of those two outcomes?

Note that once we have seen all ten coins, "we have at least $6$ heads and at least $3$ tails" is the same event as "we have either $7$ heads and $3$ tails or $6$ heads and $4$ tails."

The probability of $7$ heads and $3$ tails given that we have either $7$ heads and $3$ tails or $6$ heads and $4$ tails is what you computed in the question, namely,

$$ P(7H3T \mid 7H3T \cap 6H4T) = \frac{P(7H3T)}{P(7H3T) + P(6H4T)} = \frac{\binom{10}3 2^{-10}}{\binom{10}3 2^{-10} + \binom{10}4 2^{-10}} = \frac{4}{11}. $$

Similarly,

$$ P(6H4T\mid 7H3T \cap 6H4T) = \frac{7}{11}. $$

These are valid conclusions for this particular exercise. But note that this is not the gambler's fallacy.

The gambler's fallacy is simulated by your modified bet. The gambler has information about only nine of the coins, chosen without regard to whether they showed heads or tails. (Classically, it is the first nine flips out of ten flips, but randomly selecting nine coins out of ten gives the same probabilities.)

Since we look at only nine coins in order to conclude that there are at least $6$ heads and at least $3$ tails in the complete set of ten coins, we need to consider the prior probabilities not just for the entire set of coins, but also for the nine coins that we have seen.

Let $6H3T$ be the event that those nine coins have six heads and three tails. Also, let $7H2T$ be the event that those nine coins have seven head and two tails, and let $5H4T$ be the event that those nine coins have five heads and four tails. Then when we write $(6H3T, 6H4T)$ we mean that the nine revealed coins have six heads and three tails and the complete set has six heads and four tails, that is, the coin not yet revealed is a tail.

The event $7H3T$, whose probability we computed by considering how that event can occur when we observe all ten coins, is also the union of two disjoint events:

$$ 7H3T = (6H3T, 7H3T) \cap (7H2T, 7H3T). $$

These events have probabilities \begin{align} P(6H3T, 7H3T) &= \tbinom93 2^{-9} \tbinom10 2^{-1} = \tbinom93 2^{-10}, \\ P(7H2T, 7H3T) &= \tbinom92 2^{-9} \tbinom11 2^{-1} = \tbinom92 2^{-10}. \end{align}

Checking that we have accounted for all the probability, $$ \tbinom93 2^{-10} + \tbinom92 2^{-10} = \tbinom{10}3 2^{-10} = P(7H3T). $$

Similarly, \begin{align} 6H4T &= (6H3T, 6H4T) \cap (5H4T , 6H4T) \\ P(6H3T, 6H4T) &= \tbinom93 2^{-9} \tbinom11 2^{-1} = \tbinom93 2^{-10}, \\ P(5H4T, 6H4T) &= \tbinom94 2^{-9} \tbinom10 2^{-1} = \tbinom94 2^{-10}, \end{align} and $$ \tbinom93 2^{-10} + \tbinom94 2^{-10} = \tbinom{10}4 2^{-10} = P(6H4T). $$

Since we have seen six heads and three tails on the nine revealed coins, we know for sure that neither the event $(7H2T, 7H3T)$ nor the event $(5H4T, 6H4T)$ has occurred. So it is a fallacy to use them in a calculation as if they might have occurred.

The correct calculation uses only the events $(6H3T, 7H3T)$ and $(6H3T, 6H4T)$, which together comprise all possible outcomes of the ten coins when we have seen six heads and three tails on the nine revealed coins. The correct calculation when we initially reveal only nine coins is

$$ P(7H3T \mid 6H3T) = \frac{P(6H3T, 7H3T)}{P(6H3T, 7H3T) + P(7H2T, 7H3T)} = \frac12, $$ $$ P(6H4T \mid 6H3T) = \frac{P(6H3T, 6H4T)}{P(6H3T, 6H4T) + P(5H4T, 6H4T)} = \frac12. $$