Bayes' Rule broken?!?!

Question

This question has been driving me CRAZY for 4 days now. The question comes from the textbook "One Thousand Exercises in Probability", specifically Exercise 3 in section 1.4. The solution does not make sense! The question goes as follows:

"A man possesses five coins, two double-headed, two normal and one double-tailed. The man shuts his eyes, picks a coin at random, and tosses the coin. He opens his eyes, sees a head: what is the probability the lower face is also a head?".

The book gives an answer that is $2/3$. The solution in the book goes as follows:

Event $D$ means a coin is double-headed. Event $N$ means a coin is normal. Event $H_1^U$ means there is a head on first toss on the upper side of coin, whilst event $H_1^L$ means that there is a head on first toss on the lower side of the coin.

Probability of upper head on first throw = $\mathbb{P}{(H_1^U)}=\frac{2}{5}*\frac{1}{2}+\frac{2}{5}*1+\frac{1}{5}*0=\frac{3}{5}$

Bayes rule, reusing the result above, then goes as follows:

Probability of lower-side heads, given upper-side heads = $\mathbb{P}{(H_1^L | H_1^U)}=\frac{\mathbb{P}{(H_1^L \cap H_1^U)}}{\mathbb{P}{(H_1^U)}}=\frac{\mathbb{P}{(D)}}{\mathbb{P}{(H_1^U)}}=\frac{\frac{2}{5}}{\frac{3}{5}}=\frac{2}{3}$

Logical answer: if the man sees heads when he opens his eyes after the first toss, it means the coin must be either double-headed or normal!!! That is our probability space, given that the first toss produces heads on the upper side of the coin. Therefore, the probability that there is heads on the lower side of the coin is the probability that we have a double-headed coin, given that there are 4 possible coins we could be dealing with: two of them double-headed and two of them normal. Therefore, I see exactly 0.5 probability of lower-side being heads, given that the upper side is also heads!

Pls help me solve the logical fallacy here, otherwise I won't be able to sleep at night :). Thank you.

Answer 1

The problem is that seeing heads changes your estimate of the probability that you have seen the $HH$ or $HT$ coins.

To see this intuitively, suppose that, instead of coins, you had a pair of trillion sided dice. Say one die has all $H's$ and the other has one $H$ and all the rest $T$. You choose a die uniformly at random and toss $H$. Which die do you have? While it is possible that you have the one with a single $H$, that is astonishingly unlikely.

To the given problem, I've always thought that the easiest way to see the answer was to note that, when you toss a randomly selected coin, each side (of all the coins) is equally likely to come up. Given that you see a $H$, it's equally likely that it's any given $H$ side. There are $4$ $H$ sides that come from $HH$ coins, and $2$ that come from $HT$ coins, hence the answer is $$\frac 4{4+2}=\frac 23$$

Applying this logic to the trillion sided dice we see that the probability that you have the all $H$ die is $$\frac {10^{12}}{10^{12}+1}\approx 1-10^{-12}$$

Answer 2

The probability space is composed of heads that might be witnessed after the selection and toss.   Every head is equally likely to be witnessed.   If you like, think of each head as being labelled with a unique mark in invisible ink.

There are six heads that the witness could be viewing after the toss, and four of them have another head on the other side of their coin, while two do not.   So the probability that a double sided coin had been selected when given that a head was witnessed is $2/3$.

Answer 3

The probability that there is a head on the upper side of the first coin and a tail on the lower side is not the probability that the first coin is normal, but half as much. Indeed if the first coin is normal, then there is an equal chance that the upper side of the first coin be a head or a rail. That corrects your closing computation.

As was said in the comments, while it’s true that there are four outcomes in the probability space of coins, given that the first flip was a head, the outcomes are not all equally likely!

Answer 4

As perhaps a more simple explanation, the probability of seeing a head on the coin with two heads is much higher than the other, because there are more heads. So the fact that you saw the head gives you information. If you look at the outcomes as just heads and tails, not coins. There are 6 heads, 4 of which are on coins with two heads, so if you see a heads the probability it came from a coin with two heads, is 4 out of 6 (or 2/3).