Bayes Theorem: what is wrong in using counts instead, intuitively.

Question

I am reading about Bayes Theorem where the sample example used is: We have a red box with 2 apples and 6 oranges, and a blue box with 3 apples and 1 orange. The probability of selecting a red box and blue box is 40% and 60% respectively.

Considering the following chart:

apple  |  2  |  3  |

orange |  6  |  1  |

        red    blue

P(apple| red)=count(apples in red)/count(apples and oranges in red) = 1/4
P(apple|blue)=count(apples in blue)/count(apples and oranges in blue)=3/4
P(orange|red)=count(oranges in red)/count(apples and oranges in red)=3/4
P(orange|blue)=count(oranges in blue)/count(apples and oranges in blue)=1/4

Now, I know it's wrong but intuitively why is it wrong to calculate:

P(red|apple)=count(apples in red)/count(all the apples)=2/5
P(blue|apple)=count(apples in blue)/count(all the apples)=3/5
P(red|orange)= count(oranges in red)/count(all the oranges)=6/7
P(blue|orange)= count(oranges in blue)/count(all the oranges)=1/7

but we rather apply Bayes theorem to calculate the above conditional probabilities, like

P(red|orange)=P(orange|red)*P(red)/P(orange)=3/4*4/10*20/9=2/3

How do I identify that Bayes Theorem is the right way to calculate the probabilities for a certain problem and using the counts will put me on the wrong way. What's the intuition behind it?

Thanks!

Answer 1

There are six oranges in the red box and one orange in the blue box. Concluding from this that $P(\text{red}|\text{orange})=6/7$ would be justified if each individual orange has the same probability of being selected as any other orange.

To see what is wrong with this, suppose you had one apple and one orange in the red box, $1000$ apples and $1000$ oranges in the blue box, and $50$% probability of selecting either box. By your reasoning, whether an apple or an orange is selected, it is $1000$ times as likely to come from the blue box. Do you see why that is wrong? With this set-up, each piece of fruit in the red box has one chance in four of being selected, while each piece in the blue box has one chance in four thousand.

Answer 2

The counting intuition can be made to work, but only if you give appropriate "weight" to each count.

Once you have chosen a box, every piece of fruit within it has (presumably) an equal chance of being chosen. But before you choose the box, the red box has only a $40\%$ chance to be chosen, and then once you choose it there are $8$ pieces of fruit from which to select one, so the prior probability to select a given piece of fruit is just $5\%$ for each piece in the red box. The blue box, on the other hand, has a $60\%$ chance to be chosen, and then there are only $4$ pieces within it from which to choose, so each of those pieces has a prior probability of $15\%$ to be chosen.

You have $6$ oranges in the red box, each with a $5\%$ chance. These add up to $30\%$. The single orange in the blue box has a $15\%$ chance to be chosen. Given that we choose an orange, the odds are therefore $30$ to $15$ that it came from the red box, that is, the red box has conditional probability $30/45 = 2/3$.

Another way to put this is that when you count oranges, the orange in the blue box has three times the "weight" of each orange in the red box (because in the prior probabilities, that orange was three times as likely to be selected). So instead of the ratio being $6:1$, it is $6:3$, and the chance that the orange came from the red box is $6/9$ rather than $6/7$.

Answer 3

How do I identify that Bayes Theorem is the right way to calculate the probabilities for a certain problem and using the counts will put me on the wrong way. What's the intuition behind it?

Watch for biases in selection.   Probability is only a ratio of counts when the individual things being counted all have the same probability weighting.

$\mathsf P(\text{apple}\mid\text{red}) = \frac{\textsf{count}(\text{apple in red})}{\textsf{count}(\text{fruit in red})}$ because each particular piece of fruit in the red box has the same chance of being selected.   There's no bias among individual fruits in the red box.

However, each apple in any box does not have the same chance of being selected.   A particular apple in the blue box is more likely to be selected than a particular apple in the red box.   There's a bias amongst individuals so a ratio of counts cannot be used to measure probabilities.

$\mathsf P(\text{red}\mid \text{apple}) = \frac{\mathsf P(\text{apple}\mid \text{red})\mathsf P(\text{red})}{\mathsf P(\text{apple}\mid \text{red})\mathsf P(\text{red})+\mathsf P(\text{apple}\mid \text{blue})\mathsf P(\text{blue})}=\frac{\frac 2 8\cdot 40\%}{\frac 2 8\cdot 40\%+\frac 3 4\cdot 60\%}\neq \frac{\textsf{count}(\text{apple in red})}{\textsf{count}(\text{apples})}=\frac{2}{5}$