Conditional Probabilities In Sequential Experiments

Question

Let's assume we are given a coin with no information about its bias ($P(H)$ is unknown) or independence ($P(H|T)$ may or may not be equal to $P(H)$, we only know that $P(H) + P(T) = 1$. Let's assume we toss the coin a sufficient number of times and are asked to estimate $P(H|T)$. That is the probability of getting heads after getting tails. How should we calculate the above probability?

1. Can we group the outcomes in exclusive pairs or should we apply a rolling window of two to the sequence?

The results will be different based on the above decision. Assume, we see the following sequence: ....HHTT..... if we group HH and TT separately, the count of HT will be different than if we estimate the counts on a rolling basis (HH, HT, TT).

2. Why doesn't the law of conditional probabilities $P(A|B) =\frac{P(A, B)}{P(B)}$ apply here?

Answer 1

Hi: If you want to estimate $P(H | T)$, this is equal to (the number of times that an H occurred immediately after a T) divided by (the number of times that a T occurred). Therefore, it is equal to the number of times you get an TH sequence divided by the number of times you get a T.

so, if you had T, H, T, T, H, the estimate would be 2/3.

In your specific example with H, H, T, T, it would be 0/1 = 0 because you can't count the last T because you didn't see the one after the last T.

Answer 2

Why doesn't the law of conditional probabilities $P(A|B) =\frac{P(A, B)}{P(B)}$ apply here?

It does apply.

Assume that:

• You have a sequence of $m+1$ coin tosses.
• In the first $m$ of these coin tosses, there were $n$ T's. Here, when enumerating the number of T's, the very last coin toss is automatically ignored.
• In this sequence there were $a$ occurrences of TH and $b$ occurrences of TT, where $(a + b) = n.$ In this enumeration, if the next to last element of the sequence is a T, then the last element of the sequence is considered, when enumerating the number of occurrences of either TH or TT.

So, there were $(m)$ opportunities for the TH sequence to occur, and it occurred $(a)$ times.

Therefore, $~\displaystyle p(T,H) = \frac{a}{m}.$

Similarly, $~\displaystyle p(T,T) = \frac{b}{m}.$

Therefore, $~\displaystyle p(H|T) = \frac{p(T,H)}{p(T,H) + p(T,T)} = \frac{\frac{a}{m}}{\frac{a}{m} + \frac{b}{m}} = \frac{a}{a+b} = \frac{a}{n}.$

Further, the $~\displaystyle \frac{a}{n}~$ computation may be alternatively intuited as follows: Since the T occurred $(n)$ times, there were $(n)$ opportunities for the TH to occur, and it occurred in $a$ of these opportunities.