How to understand the difference between independent events and independent random variables?

Question

I'm trying to learn probability on my own and have recently been studying random variables. The book I'm using provides an explanation of why the criterion for event independence is different than the criterion for random variable independence but I just can't get my head around it.

"Definition 3.8.2 (Independence of many r.v.s). Random variables $X_1 , \ldots , X_n$ are independent if \begin{align} & P (X_1 \leq x_1 , \ldots , X_n \leq x_n ) \\[6pt] = {} & P (X_1 \leq x_1 ) \cdots P (X_n \leq x_n ), \text{ for all } x_1 , \ldots , x_n \in\mathbb R.\end{align} For infinitely many r.v.s, we say that they are independent if every finite subset of the r.v.s is independent. Comparing this to the criteria for independence of $n$ events, it may seem strange that the independence of $X_1 , \ldots , X_n$ requires just one equality, whereas for events we needed to verify pairwise independence for all $\binom{n}{2}$ pairs, three-way independence for all $\binom{n}{3}$ triplets, and so on. However, upon closer examination of the definition, we see that independence of r.v.s requires the equality to hold for all possible $x_1 , \ldots , x_n$ -- infinitely many conditions!"

So somehow, the criteria that each r.v. being tested for independence can take on any value and have the equality still hold allows us to infer that there is tuple-wise independence between each r.v. being tested as well, unlike the criteria for events. Can someone help illuminate this for me?

Answer 1

For independent random variables I prefer following definition: $$P(X \in A, Y \in B) = P(X \in A)P(Y \in B)$$ For $\forall A, B$ from given $\sigma$-algebra. Imho, from here is more easy to see, that pairwise independence do not imply joint independence.

Answer 2

This answer summarizes my above comments in a bit more detail.

• Exercise: Suppose $A, B, C$ are events that satisfy $P[C]=0$. Prove that $$P[A\cap B\cap C] = P[A]P[B]P[C]$$ Thus, the above equation holds whenever $P[C]=0$, even when $A$ and $B$ are not independent events.

• Let $X,Y,Z$ be random variables that satisfy $$ P[X \leq 4.5, Y\leq 9, Z\leq z] = P[X \leq 4.5]P[Y\leq 9]P[Z\leq z] \quad \forall z \in \mathbb{R}$$ Since there are infinitely many equations here, we can take $z\rightarrow\infty$ to obtain:* $$ P[X \leq 4.5, Y\leq 9] = P[X\leq 4.5]P[Y \leq 9]$$ and hence the events $\{X \leq 4.5\}$ and $\{Y\leq 9\}$ are independent.

• For events $A,B,C$, if we are told $$ P[A\cap B\cap C] = P[A]P[B]P[C]$$ then we have only one equation, there is no way to "take $c\rightarrow\infty$" to obtain $P[A\cap B] = P[A]P[B]$. (See first exercise for a case when $P[A \cap B] \neq P[A]P[B]$).

---

*Note: We are using the fact $$ \lim_{z\rightarrow\infty} P[Z\leq z] = 1$$ and more generally, if $\Omega$ is the sample space, we are using the fact $\{Z\leq z\} \nearrow \Omega$ so for any event $A$ we have $A \cap \{Z \leq z\} \nearrow A \cap \Omega$ (and of course $A \cap \Omega = A$) so by the theorem of "continuity of probability": $$ \lim_{z\rightarrow\infty} P[A \cap \{Z \leq z\}] = P[A \cap \Omega] = P[A]$$

Answer 3

The basic definition of independence of random variables $X_1,\ldots,X_n$ says \begin{align} & P (X_1 \in A_1, \ldots , X_n \in A_n ) \\[6pt] = {} & P (X_1 \in A_1 ) \cdots P (X_n \in A_n ), \\ & \text{for all (Borel-)measurable } A_1,\ldots,A_n \subseteq \mathbb R. \end{align} Clearly this implies what was stated in the question. The converse implication takes more work. It's related to the proof that the c.d.f. is enough to determine a probability distribution on (Borel-)measurable subsets of the line.