Prove that $P(HH|A) = P(H|A)\times P(H|A)$ rigorously

Question

Question: Suppose that we have 2 fair coins, coin A and coin B. If we obtained 2 heads after 2 tosses, what is the probability that the coin is A?

My attempt:

By the Bayes' Theorem, we have $$P(A|HH) = \frac{P(HH|A)\times P(A)}{P(HH|A)\times P(A) + P(HH|B)\times P(B)}$$

Here is my question.

I know that $P(HH|A)$ can be calculated by calculating $[P(H|A)]^2.$ But how to prove they are the same rigirously?

From definition of conditional probability, we should have $$P(HH|A) = \frac{P(HH\cap A)}{P(A)}.$$ But the numerator does not make sense to me as both $HH$ and $A$ come from different sample space.

Answer 1

If you have two fair coins, then they are indistinguishable. The probability that the coins is A is $1/2$. No need for any long reasoning.

If you need to answer the second question. The coins are fair, the event A give you no information (the events H and A are independent) hence $$P(HH\mid A)=\frac{P(HH\cap A)}{P(A)}=\frac{P(HH)P(A)}{P(A)}=P(HH)$$ By independence of event $$P(HH\mid A)=P(HH)=P(H)^2$$ And using the first argument again $$P(HH\mid A)=P(HH)=P(H)^2=P(H\mid A)^2$$

Answer 2

To answer your question directly:

The problem stmt includes "2 fair coins, A and B". I would argue that this statement clearly implies

$$P(HH \mid A) = 1/4, \,\,\,\,\,\,\,\,\,\,\,\, P(H \mid A) = 1/2$$

and any number of things you can say about a fair coin. Isn't that what a "A is fair" means?

---

The original question is "what is $P(A \mid HH)$?" IMHO this question is actually unanswerable!

The original stmt did not give any info about how $A$ vs $B$ is picked. Maybe I always pick $A$, or always $B$, or 70-30 favoring $B$, etc. One cannot compute the posterior $P(A \mid HH)$ when the prior $P(A)$ is not given, unless one assumes some prior. In this case the natural assumption is $P(A)=P(B)=1/2$, but it is just an assumption.

The following will show that, in this example, whatever prior you pick will be also your posterior, i.e. $P(A \mid HH) = P(A)$, even if the value is not $1/2$.

• $P(A \mid HH) = {P(A \cap HH) \over P(HH)}$

• $P(A \cap HH) = P(HH \mid A) P(A) = \frac14 P(A)$ because it is given that $A$ is fair

Now what is $P(HH)$? It is "obviously" also $\frac14$ but lets prove it rigorously using the given facts that $A$ is fair and $B$ is fair.

• $P(HH) = P(HH \mid A)P(A) + P(HH\mid B) P(B) = \frac14 P(A) + \frac14 P(B) = \frac14 $

Note that in the line above, we just needed $P(A)+P(B)=1$ but we did not need $P(A)=1/2$. So now we put everything together:

$$P(A\mid HH) = {P(A\cap HH) \over P(HH)} = {\frac14 P(A) \over \frac14} = P(A)$$

In other words, in this example, because $A$ and $B$ are identical, the posterior $P(A \mid HH)$ equals the prior $P(A)$, no matter what value the prior is. If you say the answer is $1/2$, that's just because of the natural assumption that the prior is $P(A) =1/2$ when it is not explicitly stated in the problem wording.