Why do we multiply the probability of failure in binomial distribution?

Question

For example Calculate the probability of getting exactly 50 heads and 50 tails after flipping a fair coin $100$ times. then is ${100 \choose 50}\left(\frac 12\right)^{50}\left(\frac 12\right)^{50}$ the reason that we multiply $\left(\frac 12\right)^{50}$ twice is because the first one $\left(\frac 12\right)^{50}$ is consider as $\frac 12$ probability of head $50$ as $50$ times, then $\frac 12$ multiply itself $50$ equal $\left(\frac 12\right)^{50}$ I know we need to multiply the second $\left(\frac 12\right)^{50}$ term as well although it is the failure of $50$ heads (or otherwise, when we are talking about $50$ tails.) My question is: Why do we need to multiply the probability of failure events ? ( I do notice that "exactly" always seems to appear in the question)

Answer 1

Let’s say you have an unfair coin instead, with probability $p$ of heads and $q=1-p$ of tails. What’s the probability of getting $k$ heads followed by $100-k$ tails? Well, each of those 100 individual events has to occur, and since they’re independent, the probability of this sequence of tosses is just the product of all of the individual probabilities, namely $p^kq^{100-k}$. It should be obvious that this probability is the same for any sequence of $k$ heads and $100-k$ tails, so the probability of $k$ heads and $100-k$ tails in any order is this number times the number of such sequences, that is, $\binom{100}k p^k q^{100-k}$. For a fair coin and $k=50$, this becomes $\binom{100}{50} {1\over2^{50}}{1\over2^{50}} = \binom{100}{50}\frac1{2^{100}}$.

Answer 2

TL;DR For situations with a finite number of possibilities:

• $P(A \cap B) = P(A)P(B)$ if and only if $A$ and $B$ are independent.
• $P(A \cup B) = P(A) + P(B)$ if and only if $A$ and $B$ are mutually exclusive ($A\cap B = \emptyset$).

---

When you find a calculation over a large number of events confusing, it is often useful to see what happens with a small number where you can directly calculate outcomes. Instead of 50, let's look at the case of just 4 flips. Then, we can easily list all the possible outcomes: $$\begin{array}{c|c} \text{Event} & \text{Outcome}\\ \hline S_0 & \text{TTTT}\\ S_1 & \text{TTTH}\\ S_2 & \text{TTHT}\\ S_3 & \text{TTHH} \end{array}\quad \begin{array}{c|c} \text{Event} & \text{Outcome}\\ \hline S_4 & \text{THTT}\\ S_5 & \text{THTH}\\ S_6 & \text{THHT}\\ S_7 & \text{THHH} \end{array}\quad \begin{array}{c|c} \text{Event} & \text{Outcome}\\ \hline S_8 & \text{HTTT}\\ S_9 & \text{HTTH}\\ S_A & \text{HTHT}\\ S_B & \text{HTHH} \end{array}\quad \begin{array}{c|c} \text{Event} & \text{Outcome}\\ \hline S_C & \text{HHTT}\\ S_D & \text{HHTH}\\ S_E & \text{HHHT}\\ S_F & \text{HHHH} \end{array}$$

Since the coin flips are fair, each of these events are equally likely. The events are mutually exclusive, and together they cover all possible outcomes. Since there are 16 of them, the probability of each is $1/16$. To find the probability of any event, you can simply count which of these 16 are included and divide by $16$ to get the probability.

For example, the event of getting at least 2 heads is $$H_{>1} = \{S_3, S_5, S_6, S_7, S_9, S_A, S_B, S_C, S_D, S_E, S_F\}$$ Since there are 11 elements, $P(H_{>1}) = \frac {11}{16}$

The event of exactly 2 heads is $$H_{=2} =\{S_3, S_5, S_6, S_9, S_A, S_C\}$$ and therefore $P(H_{=2}) = \frac 6{16} = \frac 38$.

The event that the first and second flips were heads is $$H_{12} = \{S_C, S_D, S_E, S_F\}$$ and so $P(H_{12}) = \frac 4{16} = \frac 14$. It should be obvious, but can also be easily verified, that the probabilities are the same for any of the pairs of flips required to be heads: $H_{13}, H_{14}, H_{23}, H_{24}, H_{34}$ all have probability $1/4$.

---

Now let's examine some independent events, and events that are not independent. It should be clear that the result of the first two flips should have no affect on the outcome of the last two flips. But we can also verify this from the table: We see that every possible outcome of the first two flips has its own subtable above. In each of those subtables, every possible outcome of the second two flips occurs exactly once. So the results of the first two flips did not influence the outcomes of the second two. Hence the events $H_{12}$ and $H_{34}$ are independent. As already noted $P(H_{12}) = P(H_{34}) = 1/4$, while $H_{12}\cap H_{34} = S_F$, so $P(H_{12}\cap H_{34}) = P(S_F) = 1/16$, exactly the product of $P(H_{12})$ and $P(H_{34})$.

Why? Well think about it. Suppose you have independent events $A$ and $B$ with probabilities of $p$ and $q$ respectively. In order for $A\cap B$ to occur, first you have to have $A$ occur (by "first" I simply mean I am considering it first, not that it must occur before $B$). That has probability $p$. So in a very large number of trials $N$, some $pN$ of them have $A$ occurring. Given that $A$ occurs (so we now have only $pN$ as our total number of trials), the probability of $B$ occurring is $q$, and therefore the number of those trials where $B$ occurs is $q(pN)$. Hence the total number of trials where both occur is about $qpN$. Dividing by the total trials gives $pq$ as probability of both. This is exactly what we see above in the 4 flips case.

---

But what happens if the events are not independent? If $A$ is not independent of $B$, then the fact that $A$ occurs affects the probability that $B$ occurs, so I no longer know what probability to multiply $pN$ by to get the successful trials for $B$. It is not $q$ any longer.

In the example, $H_{12}$ and $H_{13}$ are easily seen to not be independent: If $H_{12}$ holds, then we are in one of the events $S_C, S_D, S_E, S_F$. Of these 4 possible events, there are two events where $H_{13}$ is true: $S_E, S_F$. So given that $H_{12}$ holds, the probability of $H_{13}$ holding is $P(H_{34}|H_{12}) = 2/4 = 1/2$.

But if $H_{12}$ doesn't hold, then we have one of the events $S_0$ to $S_B$. Of these 12 events, $H_{13}$ holds only for $S_A$ and $S_B$. Thus $P(H_{13}|H_{12}^{C}) = 2/12 = 1/6$.

So the probability of $H_{13}$ occurring is not the same when $H_{12}$ occurs as it is when $H_{12}$ does not occur. This is what it means to say that $H_{12}$ and $H_{13}$ are not independent.

Now $H_{12} \cap H_{13} = \{S_E, S_F\}$. So the probability of both occuring is $\frac 2{16} = \frac 18$. This is not $P(H_{12})\times P(H_{13}) = \frac 1{16}$.

---

So you can multiply probabilities together exactly when the events are independent. When can you add probabilities together? Let's try it: We have the 6 choices for two flips to be heads: $H_{12}, H_{13}, H_{14}, H_{23}, H_{24}, H_{34}$. Each of these has probability $1/4$. If we add them together, we get a "probability" of $1.5$! Not just greater than the actual probability of at least 2 heads calculated above to be $11/16$, but even greater than the maximum possible probability of $1$. Clearly something is wrong.

Note that if $S_F$ occurs, then all 6 of the 2-heads events above occur. Note that the probability calculations for them all include a contribution from $S_F$. If we add the probabilities together, we add in this contribution 6 times. But in the calculation of $P(H_{>1})$ above, it only contributed once. This is the mistake. We are over-counting the contribution of some events - not just $S_F$, but every $S$ that contributes to more than one of the 2-head events.

Suppose I find some events that have nothing in common: If $A$ is true, then $B$ is false, and vice versa? For example, I could consider the event that exactly flips 1 and 2 are heads: $E_{12} = \{S_C\}$ and exactly 1 and 3 are heads: $E_{13} = \{S_A\}$. The event that one or the other occurs is $\{S_A, S_C\}$, with probability $\frac 2{16} = P(E_{12}) + P(E_{13})$.

So you can only add probabilities when the events don't overlap. (when there are an infinite number of possible outcomes, this is loosened a little, but not much.)

---

Which brings us to the final point. Both you and amd multiplied your probabilities by a count of $100 \choose 50$. This is not a probability, so it has nothing to do with the rule for multiplying probabilities. It actually comes from the rule for adding probabilities. If we have $k$ events $\{E_i\}_{i=1}^k$ which are mutually exclusive ($E_i \cap E_j = \emptyset$ for all $i \ne j$), then $$P(\bigcup_i E_i) = P(E_1) + P(E_2) + P(E_3) + ... + P(E_k)$$ If it is also true that all of the $E_i$ have the same probability $p$, then $$P(\bigcup_i E_i) = p + p + p + ... + p = kp$$

In your original calculation, each event was that a particular set $A$ of 50 flips were all heads, while the other 50 flips were all tails. Regardless of which flips are in $A$, this specifies the outcome of all 100 flips, so the probability is $\frac 1{2^{100}} = \frac 1{2^{50}}\frac 1{2^{50}}$. There are $100 \choose 50$ possible sets $A$, which cover all possible ways that exactly 50 flips can be heads. Further, any two such events cannot happen at the same time, since they specify different outcomes for certain flips. So they are mutually exclusive and we can say that the probability is ${100 \choose 50}\frac 1{2^{100}}$.

But you cannot do the same for the events of getting 10 heads (without specifying 90 tails), since the events of getting particular sets of 10 heads are not mutually exclusive.