How can two mutually exclusive events be dependent given that they are two results of the same experiment?

Question

I'm taking a probability and stats course and I'm very confused about the fact that mutually exclusive events are necessarily dependent.

The way I conceptualize dependence/independence is whether one event directly affects the other. For example, if it is raining, I am more likely to bring an umbrella. Or if I pick a king from a deck or cards, I'm less likely to pick another king on my next go than if I had picked a non-king as my first card. Both of these examples have a sense of events being sequential. This makes it easy for me to understand that their non-mutually-exclusive events are dependent, that is, that $P(A|B) \ne P(A).$

However, consider the rolling of a die, where rolling an even number is event $A,$ and rolling an odd number is event $B.$ These events are clearly mutually exclusive, but I can't wrap my head around why they're also dependent. I don't get how the aforementioned formula works. How could you ever calculate the probability of the event of rolling an even number given the fact that you've rolled an odd number, if they're happening at the exact same time? How can two mutually exclusive events be dependent given that they are two results of the same experiment?

Perhaps this is just something that I'm looking at the wrong way, but I can't grasp the concept of events being dependent on each other without being sequential in some way. Has anyone else experienced this sort of confusion or have a way to grasp the concept?

Answer 1

I can't grasp the concept of events being dependent on each other without being sequential in some way.

Let's temporarily forget about independence and mutual exclusivity, and first address the pressing fundamental error in your conceptualisation. Now, consider a probability experiment with sample space $\{ttt,htt,tht,tth,hht,hth,thh,hhh\}.$ Note that an event is literally a subset of the sample space:

• the event of obtaining exactly one Head is $\{htt,tht,tth\}$
• the event of obtaining fewer than four Heads is the sample space itself
• the event of obtaining four Heads is $\emptyset.$

Observe from the above that it does not generally make sense to frame events as existing at timepoints, let alone as being in sequence. After all, every event is just a specific ‘snapshot’ of associated outcomes! Elaboration here and here. By ‘sequential’, you are thinking only of events like

• the event of obtaining Tail in the 1st toss $=\{thh,tht,tth,ttt\}$
• the event of obtaining Tail in the 2nd toss $=\{ttt,htt,tth,hth\}.$

---

The way I conceptualize dependence/independence is whether one event directly affects the other.

As is clear by now, this framing works only for a strict subset of all the possible cases. An accurate characterisation of independence—corresponding to $P(B|A) = P(B)$ —is this:

• given event $A$ with a nonzero probability and event $B,$ they are independent iff knowing that $A$ happens does not change $B$'s probability.

This characterisation (applicable only when $A$ has a nonzero probability) is actually a consequence of the following definition (applicable regardless of $A$'s probability):

• $A$ and $B$ are independent events $\iff P(A∩B)=P(A)P(B).$

---

if it is raining, I am more likely to bring an umbrella.

event of raining $=\{\color{brown}{ru},r\overline u\}$
event of bringing umbrella $=\{\color{brown}{ru},\overline ru\}$

rolling an even number is event $A,$ and rolling an odd number is event $B.$

$A=$ event of obtaining even $=\{2,4,6\}$
$B=$ event of obtaining odd $=\{1,3,5\}$
$C=$ event of obtaining multiple of $3$ $=\{3,6\}$
$D=$ event of obtaining multiple of $5$ $=\{5\}$

By definition,

• $A$ and $B$ are dependent
• $B$ and $C$ are independent
• $B$ and $D$ are dependent.

Depending on how you are intuiting the concept of independence, you may be surprised by some of the above bullets!

---

How could you ever calculate the probability of the event of rolling an even number given the fact that you've rolled an odd number, if they're happening at the exact same time?

By definition! $P(A|B)=\dfrac{P(A\cap B)}{P(B)}=0.$

I'm very confused about the fact that two mutually exclusive events are necessarily dependent.

More accurately,

• two mutually exclusive events are dependent if and only if they have nonzero probabilities.

This is corroborated by the definitions of independence and mutual exclusivity. More concretely, suppose that event $A$ is Tail and event $B$ is neither Head nor Tail. Knowing that $A$ happens does not change $B$'s zero probability, so $A$ and $B$ are independent; yet $A$ and $B$ are also mutually exclusive.

Answer 2

Your conceptualization isn't quite right. The way to conceptualize independence is by asking whether knowledge of one event occurring affects the probability of another event occurring.

Formally, we can make this notion rigorous with the notion of conditional probability. We say $A$ and $B$ are independent if the probability that $A$ happens in a vaccuum is the same as the probability that it happens under the restriction that $B$ happens. Formally, this means $$P(A)=P(A|B)\text{.}\tag{1}$$

In your die example, in a vaccuum, the probability of the die coming up odd is $\frac{1}{2}$, but if you know that the die comes up even, then all of a sudden your probability for whether the die comes up odd becomes $0$, as you are certain it does not happen. So this violates the technical definition of independence, since $$P(\text{odd}\mid\text{even})=0\neq \frac{1}{2}=P(\text{odd})\text{,}$$ but it should violate your intuition about independence as well, since the probability of an odd roll happening is affected by (i.e., "depends on") the restriction that an even roll happens.

Addendum

As Rob Arthan points out in the comments, one can also define independence of two events as $$P(A\cap B)=P(A)P(B)\text{.}\tag{2}$$ This definition is technically the correct one, and is slightly more general, since it allows either $A$ or $B$ to have $0$ probability. Interestingly enough, under this interpretation technically two mutually exclusive events can be independent, provided that one of them has $0$ probability.

Most of the time, i.e., when $B$ has positive probability, dividing both sides of (2) by $P(B)$ and using the definition of conditional probability yields the characterization I gave in equation (1) above.