What is the probability of drawing a blue ball on the 3rd trial (without replacement)?

Question

I am self-studying Professor Ery Arias-Castro's Principles of Statistical Analysis (it's a pretty great book so far), and I could use some help on this problem.

Suppose we draw without replacement from an urn with $r$ red balls and $b$ blue balls. At each stage, every ball remaining in the urn is equally likely to be picked. Use (1.15) to derive the probability of drawing a blue ball on the 3rd trial.

Equation 1.15 is

$$ \mathbb{P}(A) = \mathbb{P}(A | B) \mathbb{P}(B) + \mathbb{P}(A | B^\complement) \mathbb{P}(B^\complement). $$

Using the notation $B_n$/$R_n$ for the event of drawing a blue/red ball on the $n$th trial, I calculated

\begin{align} \mathbb{P}(B_2) &= \mathbb{P}(B_2 | B_1) \mathbb{P}(B_1) + \mathbb{P}(B_2 | R_1) \mathbb{P}(R_1)\\ &= \frac{b - 1}{b + r -1} \frac{b}{b + r} + \frac{b}{b + r - 1} \frac{r}{b + r}\\ &= \frac{b}{b + r}. \end{align}

Similarly, we get $\mathbb{P}(R_2) = r / (b + r)$, but I am having a hard time setting up the equation for $\mathbb{P}(B_3)$. I would greatly appreciate some help.

Answer 1

I'd suppose that, at some point, you'll need the following extension of the equation you wrote: $$ P(A|B) = P(A|B,C)P(C|B) + P(A|B,C^c)P(C^c|B).$$ You can prove this the same way that you proved the original statement.

With this in hand, $$ P(B_3) = P(B_3|R_1) P(R_1) + P(B_3|B_1) P(B_1).$$ But $$ P(B_3|R_1) = P(B_3|R_1, R_2) P(R_2|R_1) + P(B_3|R_1,B_2) P(B_2|R_1), $$ and everything in the right hand side should be straightforward to compute. Similarly, you can expand $P(B_3|B_1)$, and then merge the answers.

---

Incidentally, for sampling without replacement, there's an easier way to compute things: think of the sample space as the entire sequence of $N = b+r$ length strings composed of either $B$ or $R$. Then the sampling without replacement process will end up producing one of these strings, where each string is equally likely.

Now, the total number of such strings is $\binom{N}{b}$ - take $N$ slots, and pick $b$ of them to be blue. The number of strings where the third symbol is a $B$ is instead $\binom{N-1}{b-1}$: place one $B$ in the third slot, and choose $b-1$ out of the remaining $N-1$ slots to put $B$s in. So the probability is $$ \binom{N-1}{b-1}/\binom{N}{b} = \frac{b}{N} = \frac{b}{b+r}.$$ Note that the answer doesn't change if you replace "third entry is $B$" by "$k$th entry is $B$" for any $k \le n+r$.

Answer 2

My detailed solution to the problem. Thank you to @stochasticboy321 for helping me clarify the ideas.

First, we note the general law of total probability in the discrete case.

Theorem (Law of Total Probability): Let $(\Omega, \Sigma, \mathbb{P})$ be a probability space, and suppose $\{B_n\}_{n=1}^\infty \subseteq \Sigma$ is a countable collection of mutually exclusive ($B_i \cap B_j = \varnothing$ for $i \neq j$) and collectively exhaustive ($\cup_{n = 1}^\infty B_n = \Omega$) events. Then, for any $A \in \Sigma$, we have

\begin{align} \mathbb{P}(A) &= \sum_{n=1}^\infty(A \cap B_n)\\ &= \sum_{n=1}^\infty\mathbb{P}(A | B_n)\mathbb{P}(B_n). \end{align}

In the problem, our experiment is drawing balls out of an urn without replacement until we empty the urn. Our sample space $\Omega$ consists of words of length $b+r$ in two letters $B$ and $R$. Let $B_k$ and $R_k$ represent the event of choosing a blue or red ball on the $k$th draw respectively.

We note that

$$ (B_2 \cap B_1) \cup (B_2 \cap R_1) \cup (R_2 \cap B_1) \cup (R_2 \cap R_1) = \Omega $$

and that any two of these events are disjoint (their intersection is empty). Thus, by the law of total probability, we have

\begin{align} \mathbb{P}(B_3) &= \mathbb{P}(B_3 | B_2 \cap B_1) \mathbb{P}(B_2 \cap B_1) + \mathbb{P}(B_3 | B_2 \cap R_1) \mathbb{P}(B_2 \cap R_1) + \mathbb{P}(B_3 | R_2 \cap B_1) \mathbb{P}(R_2 \cap B_1) + \mathbb{P}(B_3 | R_2 \cap R_1) \mathbb{P}(R_2 \cap R_1)\\ &= \mathbb{P}(B_3 | B_2 \cap B_1) \mathbb{P}(B_2 | B_1) \mathbb{P}(B_1) + \mathbb{P}(B_3 | B_2 \cap R_1) \mathbb{P}(B_2 | R_1) \mathbb{P}(R_1) + \mathbb{P}(B_3 | R_2 \cap B_1) \mathbb{P}(R_2 | B_1) \mathbb{P}(B_1) + \mathbb{P}(B_3 | R_2 \cap R_1) \mathbb{P}(R_2 | R_1) \mathbb{P}(R_1)\\ &= \frac{b - 2}{b + r - 2} \frac{b - 1}{b + r -1} \frac{b}{b + r} + \frac{b - 1}{b + r - 2} \frac{b}{b + r -1} \frac{r}{b + r} + \frac{b - 1}{b + r - 2} \frac{r}{b + r -1} \frac{b}{b + r} + \frac{b}{b + r - 2} \frac{r - 1}{b + r -1} \frac{r}{b + r}\\ &= \frac{b(b^2 + r^2 + 2br - 3b -3r + 2)}{(b+r)(b^2 + r^2 + 2br - 3b -3r + 2)}\\ &= \frac{b}{b + r}. \end{align}

Remark: As noted by @stochasticboy321, you could solve this problem in a combinatorial fashion as shown in their solution, but this solution uses the law of total probability as requested.