Confusion on Independent Probabilities and Notation

Question

Suppose we have a fair coin. The event Heads will be marked as $H$ and Tails as $T$. So we have, for an arbitrary flip

$$P(H) = 1/2 \\ P(T) = 1/2$$

We wish to calculate the probability of flipping the coin twice and obtaining either

• both tails
• both heads
• one heads, one tails $(\star)$

Now, some of my confusion begins. I'll try to keep a numbered list of the concerns I have.

1. The problem as stated is ambiguous. For the bullet marked by $(\star)$ it's unclear whether I mean the sequence $(H, T)$, with respect to time, or the unordered pair $\{T, H\}$. Clearly these things have different probabilities.

Regarding my confusion in (1), the author of the question could easily make precise what is meant (as I did in my statement of confusion). However, the original ambiguity leads me to what I think is my deeper confusion on the definition of the probability of independent events. For example—and by Wikipedia's definition—if events $A$ and $B$ are independent, then their joint probability is the product of their individual probabilities. In other words:

$$P\left(A\cap B\right) = P\left(A\right)P\left(B\right)$$

So, this definition seems to correspond to the probability of flipping our coin and getting one of the sequences: $TH$, $HT$. Or we could discard time considerations and consider $HT = TH$, more formally in set notation as $\{T, H\} = \{H, T\}$, then add $P(HT) + P(TH)$ to get $P(H\cap T)$...?

2. I think I should normally interpret independent probability questions like this as irrespective of time or sequence. Except in the sense that the number of sequences that are "unordered equivalent" to the unordered pair in question, are the "weight" of the unordered pair.

Okay, that might have been a confusing but it's because I'm confused. Maybe what I mean can be made clear by building what I think is the full situation of the coin flipping, up to two flips.

\begin{align} P(T) &= 0.5\\ P(H) &= 0.5 \\ P(TT) = P(T \cap T) &= 0.25 \\ P(HH) = P(H \cap H) &= 0.25 \\ P(HT) = P(H)P(T) = P(T)P(H) = P(TH) &=0.25 \\ P(T \cap H) = P(HT) + P(TH) &=0.5 \end{align}

Where concatenated letters refer to the sequence and the set operator refers to the unordered pair (or set).

3. Is this notation consistent?

And lastly, I was motivated to ask these questions primarily by a problem I read in Data Science from Scratch, which essentially asks the follow up question: what is the probability of getting two Heads given that the first flip was Heads?

The answer is intuitively $0.5$—the problem becomes only dependent on the second, yet-to-be-flipped coin, but how does the notation work? To set it up, based on the very definition provided by the text, we have

\begin{align} P(H \cap H | H) = \frac{P \left((H \cap H)\cap H\right)}{P(H)} = \frac{P(H)P(H)P(H)}{P(H)} \end{align}

Which doesn't make sense, because there is not third flip of the coin, and the second flip is independent of the first. But if we think of the coin flipping, and conditional probability in terms of a sequence we could write

\begin{align} P(HH | H) = \frac{P \left((HH)\cap H\right)}{P(H)} = \frac{P(H)P(H)P(H)}{P(H)} \end{align}

But ends up being the same thing?

3. I'm missing something here, whether it's notation, definition(s), or conceptually. Will you please guide me, in a rigorous way, to the intuitive result that: given one heads, the probability of getting two is $1/2$?

Thank you!

Answer 1

There are really $4$ events (at least): $H_1, T_1, H_2, T_2$ where the subscript denotes whether you're talking about the first or second coin. In coin flipping problems the subscripts are normally dropped because they are obvious from context. Once they become non-obvious then you have to conceptually add them back.

The notation $A \cap B$ means the two events both happen (conjunction). So e.g. $P(H_1 \cap T_2) = 1/4$ but $P(H_1 \cap T_1) = 0$.

The notations $HT$ or $(H,T)$ normally denotes $H_1 \cap T_2$. Similarly $HH = H_1 \cap H_2$. The subscripts are inferred by the position in the sequence.

The notation $A \cap A$ normally just means $A$ (this is from set theory). In the context of flipping multiple coins, I would avoid things like $H \cap H$ because (1) by normal rules of math $H \cap H = H$ but (2) by context of the problem it might mean $H\cap H = HH = H_1 \cap H_2$.

Anyway to answer your question 3: The correct equation is:

$$P(H_1 \cap H_2 | H_1) = {P(H_1 \cap H_2 \cap H_1) \over P(H_1)} = {P(H_1 \cap H_2) \over P(H_1)} = 1/2$$

where each of my $\cap$ means conjunction. However, when you use $(H \cap H) \cap H$, your first $\cap$ means sequence but your second $\cap$ means conjunction, hence the confusion. I mean, how could you or anyone else tell that in your $(H \cap H) \cap H$ the first and third $H$'s are the same but the middle one is different?

Hope this helps?