Binomial random variable with coin flips. How do we get the formula $P(X=h) = \binom{n}{h}p^h(1-p)^{n-h}$?

Question

Let's say a coin produces $H$ (for heads) with probability $p$ (and thus it produces $T$ with probability $p-1$. Let $X$ denote the number of $H$s we see and let $h$ be an integer. Then, if we flip the coin $n$ times:

$P(X=h) = \binom{n}{h}p^h(1-p)^{n-h}$

I'm not quite understanding where this formula is coming from.

I can see that there are $\binom{n}{h}$ ways to choose a $h$-length subset of an $n$-length set (i.e. there are $\binom{n}{h}$ $n$-length sequences where we see $h$ heads), but why do we multiply that with the probability? Furthermore, why do we multiply the probabilities of heads and tails together?

I get that coin flipping is an independent repeated trial, but I'm not sure how to apply that here (or whether it's even relevant). Any help is appreciated!

Answer 1

I can see that there are $\binom{n}{h}$ ways to choose a $h$-length subset of an $n$-length set (i.e. there are $\binom{n}{h}$ $n$-length sequences where we see $h$ heads),

Say, rather, $\tbinom n h$ is the count of ways to select $h$ elements from a set of $n$ distinct elements.   For example, the ways to select $h$ positions from a sequence of $n$ trials.

Thus $\tbinom nh$ is the count of ways to arrange $h$ heads among $n$ trial results.

but why do we multiply that with the probability? Furthermore, why do we multiply the probabilities of heads and tails together?

$p$ is the probability that a trial will result in heads.   $p^h$ is the probability that $h$ independent trials will do so.   $(1-p)^{n-h}$ is the probability that $n-h$ independent trials will result in tails.   $p^h(1-p)^{n-h}$ is therefore the probability that the first $h$ independent trials will result in heads and the next $n-h$ independent trials will result in tails.

However that is for a particular arrangement of $h$ heads and $n-h$ tails.   We want the probability for any arrangement of that many heads and tails among $n$ trials.   So we must multiply by the count of such arrangements (since all arrangements are equally probable).$$\mathsf P(H{=}h) ~=~\dbinom n h p^h(1-p)^{(n-h)}\qquad\big[h\in\Bbb N, 0\leq h\leq n\big]$$

Answer 2

The flips are independent, each with probability $p$ of being a head or $1-p$ of being a tail. Each of the $\binom{n}{h}$ choices of $h$ flips to be heads gives a head-tail sequence. Its probability is a product of $p$ and $1-p$ factors for heads and tails respectively, i.e. $p^h(1-p)^{n-h}$. (Or in @N.F.Taussig's words: "Since each of the $h$ heads has probability $p$ of occurring and each of the $n−h$ tails has probability $1−p$ of occurring, the probability that a particular sequence of $h$ heads and $n−h$ tails will occur is $p^h(1−p)^{n−h}$.") Together, these have probability $\binom{n}{h}p^h(1-p)^{n-h}$. As a sanity check,$$\sum_{h=0}^n\binom{n}{h}p^h(1-p)^{n-h}=(p+1-p)^n=1^n=1$$by the binomial theorem, so probabilities sum to $1$ as expected.

Answer 3

After one throw, $H$ with probability $p$ and $T$ with probability $q$. In other words, $$1H: p, 0H: q.$$

After two throws, $HH$ with probability $pp$, $HT$ with probability $pq$, $TH$ with probability $qp$ and $TT$ with probability $qq$. In other words, $$2H: p^2, 1H: 2pq, 0H: q^2.$$

After three throws, $HHH$ with probability $p^3$, $HTH$ with probability $pqp$, $THH$ with probability $qpp$, $TTH$ with probability $q^2p$, $HHT$ with probability $p^2q$, $HTT$ with probability $pqq$, $THT$ with probability $qpq$ and $TTT$ with probability $qqq$. In other words, $$3H: p^3, 2H: 3p^2q, 1H: 3pq^2, 0H: q^3.$$

Answer 4

You know that the binomial coefficient counts the ways a h-length subset of an n-length set.

Now we can calculate the probability that we flip $h$ heads first and then $n-h$ tails (one way):

$$\underbrace{111...111}_{\text{h times}}\underbrace{000...000}_{\text{n-h times}}$$

The probability $p^h\cdot (1-p)^{n-h}\quad (\color{blue}*)$.

In general we have $n$ independent Bernoulli distributed variables $Y_i$. So we calculate that

$$P(Y_1=1\cap Y_2=1 \cap \ldots \cap Y_h=1 \cap Y_{h+1}=0 \cap \ldots \cap Y_n=0)$$ Since $Y_i$´s are independent it is equal to $$P(Y_1=1)\cdot P(Y_2=1)\cdot \ldots \cdot P(Y_{h}=1)\cdot P(Y_{h+1}=0)\cdot \ldots \cdot P(Y_{n}=0)$$

With $P(Y_i=1)=p$ and $P(Y_i=0)=1-p$ we obtain $(\color{blue}*)$.

Answer 5

I get that coin flipping is an independent repeated trial, but I'm not sure how to apply that here (or whether it's even relevant).

Yes, the independence of trials is significant here (in binomial or miltinomial experiments).

If you flip a coin $n=1$ time, the possible outcomes are: $H$ or $T$. Let $X$ show the number of heads. Then: $$P(X=0)=P(T)=\frac1q; \\ P(X=1)=P(H)=\frac1p.$$ If you flip a coin $n=2$ times, the possible outcomes are: $HH,HT,TH,TT$. Then: $$\begin{align}P(X=0)&=P(T_1T_2)=P(T_1)\cdot P(T_2)=\frac1q\cdot \frac1q={2\choose 0}\left(\frac1p\right)^0\left(\frac1{q}\right)^2\\ P(X=1)&=P(H_1T_2)+P(T_1H_2)=P(H_1)\cdot P(T_2)+P(T_1)\cdot P(H_2)=\\ &=\frac1p\cdot \frac1q+\frac1q\cdot \frac1p={2\choose 1}\left(\frac1p\right)^1\left(\frac1{q}\right)^1\\ P(X=2)&=P(H_1H_2)=P(H_1)\cdot P(H_2)=\frac1p\cdot \frac1p={2\choose 2}\left(\frac1p\right)^2\left(\frac1{q}\right)^0\end{align}$$ Note: Because the trials are independent, $P(T_1T_2)=P(T_1)\cdot P(T_2|T_1)=P(T_1)\cdot P(T_2)$, that is, $P(T_2|T_1)=P(T_2)$, which is the condition for independent events.

I see you understand how to find the number of outcomes with $h$ heads using the combination formula.