When is P(A and B) = P(A) * P(B) in the conditional probability formula?

Question

P (A and B) = P(A) + P(B) - P(A or B)

If independent, P(A and B) = P(A) * P(B)

Why is it that we often use P(A)*P(B) for P(A and B), even when the events aren't independent (e.g. no replacement)?

For instance, in the conditional probability formula, I've found this dilemma occur often when I'm figuring out P(A and B).

How do we know when to multiply and when to use the original 'and' formula, which consists of addition and subtraction of a probabilities?

Edit: An example is a jar with 3 blue marbles and 4 yellow marbles. You pick one out randomly and do not replace. You pick another marble. What's the probability that you get two yellow marbles?

I think I would do (4/7) * (3/6). P(A and B) is found by multiplying the two.

This is one example I just made up. I've come across many problems like this and wondered this same question -- how do I know when to multiply and when to use th original formula?

Answer 1

I think I would do (4/7) * (3/6). P(A and B) is found by multiplying the two.

The key thing here is that you are getting the probability $\frac36$ by assuming there are only $6$ marbles remaining and $3$ of them are yellow. That is, you have found a probability of drawing a yellow marble on the second draw that is correct only if the first marble drawn was yellow, and not in any other case. That is practically the intuitive meaning of "given $A$" in this example.

That is, what you have found is \begin{align} P(A) &= \frac47, \\ P(B\mid A) &= \frac36 = \frac12, \\ P(A \cap B) &= P(A) P(B\mid A) = \frac27. \end{align}

On the other hand, if $A^\complement$ is the event that your first marble is not yellow, then after drawing a first marble that is blue (not yellow) you have $6$ remaining marbles of which $4$ are yellow, giving you a $\frac46$ chance to draw yellow on the second draw if the first draw is not yellow:

$$ P(B\mid A^\complement) = \frac46 = \frac23.$$

Therefore

$$ P(A^\complement \cap B) = P(A^\complement) P(B\mid A^\complement) = \frac37 \times \frac23 = \frac27. $$

We can now compute $P(B),$ the probability of drawing a yellow marble the second time not assuming anything about which marble was drawn the first time. There are two ways for $B$ to happen: draw yellow the first time and yellow the second time, or draw blue the first time and yellow the second time. It is impossible for both these sequences to happen, so we can simply add their probabilities:

$$ P(B) = P(A\cap B) + P(A^\complement \cap B) = \frac27 + \frac37 \times \frac46 = \frac47. $$

Surprised? We have $P(B) = P(A).$ After all, you are simply drawing two marbles. Why should any particular marble have a better chance than any other marble to be the second marble drawn? They all have the same chance, $\frac17,$ and the chances of the $4$ yellow marbles add up to $\frac47.$

So we see that indeed

$$ P(A \cap B) = \frac27 \neq \frac{16}{49} = P(A) P(B). $$

The probabilities are unequal and the events are not independent.

Answer 2

See Matthew Towers subsequent comment, which indicates that the analysis in this answer assumes that $p(A) \neq 0 \neq p(B).$

---

The rule is that regardless of whether A and B are independent events,
$p(A|B) \times p(B) = p(AB) = p(B|A) \times p(A).$

By definition, A and B are independent events if and only if
$p(A|B) = p(A).$

Note, it immediately follows (from this answer's first equation) that $p(A|B) = p(A)$ if and only if $p(B|A) = p(B).$