How to start with Bayes theorem

Question

I'm just learning about Bayes, I don't have any Math background at all, so this is really a newbie question. I know there are lots of questions about Bayes here already but most involve some kind of added complexity on top of the basic formula.

So my question: I have a small stack of three cards, about which I have the following information:

• The small stack has three cards drawn from a larger stack that has 1/3 red cards, 1/3 green cards, and 1/3 orange cards - (let's say the large stack has 99 cards).

So, I would reason that if you have ONLY this information then the odds of one card being green is 1/3.

• From this small stack I have picked a random card three times. Before picking, the cards were shuffled. After viewing, the picked card is put back in - so the small stack always has three shuffled cards. My three picks all came up with a green card.

Having these test results should allow me to make a more precise prediction using Bayes Theorem - I think... ?

• A (event) all cards are green
• B (test) three random picks all came up with a green card

So the formula

$$P(A|B) = \frac{P(B|A) P(A)}{P(B)}$$

becomes

$$P(A|B) = \frac{P(B|A) 1/3}{1/27}$$

because

$P(A) = 1/3$

$P(B) = 1/27$

But what is $P(B|A)$ ? Does it mean that in the event of drawing three random green cards, what are the odds that all cards are green? How do I calculate that?

and also, if I know $P(B|A)$ why do I even need the rest of the theorem... :)

Answer 1

You've made couple of mistakes.

First of all, $P(A) \neq 1/3$ in general. This probability depends on how big original stack is. Since you now fixed that size of the big stack is $99$, then there are $33$ green cards, so $$P(A) = \frac{33}{99}\cdot\frac{32}{98}\cdot\frac{31}{97}.$$

Furthermore, $P(B) \neq 1/27$. This would work if you knew beforehand that the small stack contains exactly $1$ green card. Obviously, you do not know this. I will return to how to calculate this properly.

Finally, $P(B|A)$ is actually the easiest one. The question is: "What is the probability that I will pick green card three times in a row if all the cards in stack are green?" and the answer is $100\%$.

To solve the problem, let me change your notation a bit, let $A_i$ = "there are $i$ green cards in small stack". Now, you want to calculate $P(A_3|B)$:

$$P(A_3|B) = \frac{P(B|A_3)P(A_3)}{P(B)} = \frac{P(A_3)}{P(B)}$$ because, as I've said, $P(B|A_3) = 1$.

$P(A_3)$ is to be calculated as demonstrated above. The question is what to do with $P(B)$. We will use law of total probability:

$$P(B) = P(B|A_0)P(A_0)+P(B|A_1)P(A_1)+P(B|A_2)P(A_2)+P(B|A_3)P(A_3).\tag{1}$$

$P(B|A_i)$ is easy to compute, if out of $3$ cards in small stack $i$ of them are green, then the probability to draw a green card is $i/3$ and hence, probability of drawing three green cards in a row is $$P(B|A_i)=(i/3)^3.$$ We can now see that $P(B|A_0) = 0$, so we can remove it from $(1)$, i.e., we don't have to calculate $P(A_0)$. Let us calculate $P(A_i)$ for $i>0$:

\begin{align} P(A_1) &= \frac{33}{99}\cdot\frac{66}{98}\cdot\frac{65}{97}+ \frac{66}{99}\cdot\frac{33}{98}\cdot\frac{65}{97}+\frac{66}{99}\cdot\frac{65}{98}\cdot\frac{33}{97} = 3\cdot\frac{33}{99}\cdot\frac{66\cdot 65}{98\cdot 97} = \frac{66\cdot 65}{98\cdot 97}\\ P(A_2) &= \frac{33}{99}\cdot\frac{32}{98}\cdot\frac{66}{97}+ \frac{33}{99}\cdot\frac{66}{98}\cdot\frac{32}{97}+\frac{66}{99}\cdot\frac{33}{98}\cdot\frac{32}{97} = 3\cdot\frac{33}{99}\cdot\frac{66\cdot 32}{98\cdot 97} = \frac{66\cdot 32}{98\cdot 97}\\ P(A_3) &= \frac{33}{99}\cdot\frac{32}{98}\cdot\frac{31}{97} = \frac 13\cdot\frac{32\cdot 31}{98\cdot 97} \end{align} and thus $$P(B) = \left( \frac 13 \right)^3\frac{66\cdot 65}{98\cdot 97}+ \left( \frac 23 \right)^3\frac{66\cdot 32}{98\cdot 97}+\left( \frac 33 \right)^3\frac 13\cdot\frac{32\cdot 32}{98\cdot 97} = \frac{239}{2037}$$

and finally,

$$P(A_3|B) = \frac{\frac{496}{14259}}{\frac{239}{2037}} = \frac{496}{1673}\approx 29.6473\%.$$

Compare this number to $P(A_3) \approx 3.4785\%$. And if instead of $3$ times you happened to make $k$ random picks and it turned green every time, then you would have probability $$P(B_k) = \left(\frac 13\right)^kP(A_1) + \left(\frac 23\right)^kP(A_2)+\left(\frac 33\right)^kP(A_3)\to P(A_3),\ \text{when}\ k\to\infty$$ and thus $P(A_3|B_k)$ converges to $100\%$ as $k\to\infty$. This shouldn't be surprising, the more testing we do, the more accurate our predictions are, and if all we ever get are green cards, we can be pretty sure that all three cards are green.

Here's a table that demonstrates how sure we would be that all three cards in small stack were green given that we picked $k$ green cards in a row:

\begin{array}{c|c} k& P(A_3|B_k)\\ \hline 1& 10.4355\%\\ 3& 29.6473\%\\ 5& 52.7847\%\\ 7& 72.4767\%\\ 10& 90.0105\%\\ 15& 98.5623\%\\ 20& 99.8083\%\\ 30& 99.9967\% \end{array}

Answer 2

Your definitions do not really make sense here. You say that

A - Event that next card is green B - Event that all three cards are green

But what is then P(A|B) ? You have a stack of three cards, and want to calculate the probability that the next card is green given that all three cards in the stack are green. But since we have already checked all the cards, there are no cards left to check.

Also, we can not solve this problem because you have not defined how many cards were in the original deck. It is not enough to say that there is 1/3 of each card. With a simple example: a deck of 3 cards, one of each, has a 1/3 of each distribution. But so does a 30card deck with 10 of each. But obviously in the one-of-each deck, if we pull one card, we can not pull it again. The same problem arises in your calculations. There are different probabilities that the next card is green, given that we pulled a green, depending on the number of cards in the original deck. If we use the 30deck example, the first card is green with

P(A) = 10/30 = 1/3

but the next card is only green with (if the first card was green)

P(A) = 9/29 < 1/3

Finally, the probability that you get three green cards is not P(B) = 1/9. Even if we had an infinite deck so that the probability remains as 1/3 even when we pull cards, the probability would be

1/3 * 1/3 * 1/3 = 1/27

Answer 3

Remember probability is talking about sets. $P(A)$ is the probability of the set $A$ happening, and the conditional probability of $P(B|A)$ is the probability of the set $B$ happening "inside the set" $A$.

Hence the set leading to the conditional probability is always a "smaller set" than the original set in which we are placing the condition.

Recall the formal definition: $$ P(B|A)=\frac{P(A \cap B)}{P(A)} $$

This means the conditional probability is smaller than the new universe conditioning $A$ by a factor of $P(A\cup B)$

Hence $P(B|A)<P(A)$ always.

And with the given figures: $P(A|B)={\frac 13 \over \frac 19 }P(B|A)=3P(B|A)<3P(A)=\frac 13$.

So in every case, the conditional probability will always be less than one.

For the given example: $$ P(A)=P(c_4 \text{green})=\frac 13\\ P(B)=P(c_{1,2,3} \text{green}) =\frac 1{3^3}=\frac 1 {27}\\ P(A\cup B)=P(c_{1,2,3,4}\text{green})\frac 1{3^4}=\frac 1 {81} $$

So, finally: $$ P(A|B)=P(B|A){P(A) \over P(B)}={P(A\cup B) \over P(B)} = {\frac 1{81}\over \frac 1 {27}}=\frac 13 $$