About Weak union of conditional independence

294 Views Asked by At

Problem is:

Let $(X,Y,W,Z)$ be disjoint sets of random variables each with finite space. Prove that :

If: $X$ is independent of $(Y,Z)$ given $W$

then we have : $X$ is independent of $Y$ given $Z,W$

I found a solution on conditional-independence-property-weak-union

My question is why the following proof gets an incorrect result(assume it is positive):

$P\left( {\left. {X,Y} \right|W,Z} \right) = \frac{{P\left( {\left. {X,Y,Z} \right|W} \right)}}{{P\left( {\left. Z \right|W} \right)}} = \frac{{P\left( {\left. X \right|W} \right)P\left( {\left. {Y,Z} \right|W} \right)}}{{P\left( {\left. Z \right|W} \right)}} = P\left( {\left. Y \right|Z,W} \right)P\left( {\left. X \right|W} \right) \ne P\left( {\left. Y \right|Z,W} \right)P\left( {\left. X \right|Z,W} \right)$

1

There are 1 best solutions below

1
On BEST ANSWER

Remember, the conditional probability is a function, not a number, so you need to be careful when manipulating these quantities. I will use the same interpretation of the question as David Moews did in his answer that you linked to. Thus, $W,X,Y,Z$ are random variables defined on the same finite probability space.

So the notation $\mathbb P(X,Y\mid W,Z)$ is shorthand for: the function $f(w,x,y,z)$ given by $$ f(w,x,y,z)=\mathbb P(X=x,Y=y\mid W=w, Z=z)=\frac{\mathbb P(W=w,X=x,Y=y,Z=z)}{\mathbb P(W=w,Z=z)}, $$ where $w$ and $z$ range over all values such that $\mathbb P(W=w,Z=z)>0$. Similarly, the notation $\mathbb P(X,Y,Z\mid W)$ is shorthand for the function $$ \frac{\mathbb P(W=w,X=x,Y=y,Z=z)}{\mathbb P(W=w)}, $$ with a similar caveat on the value of $w$. (I will stop mentioning this for the rest of this posting.) Continuing in this way, we can write the given condition $$ \mathbb P(X,Y,Z\mid W)=\mathbb P(X\mid W)\mathbb P(Y,Z\mid W) $$ explicitly as follows: $$ \frac{\mathbb P(W=w,X=x,Y=y,Z=z)}{\mathbb P(W=w)}=\frac{\mathbb P(W=w,X=x)}{\mathbb P(W=w)}\frac{\mathbb P(W=w,Y=y,Z=z)}{\mathbb P(W=w)}. $$ Summing both sides over all $y$ yields $$ \frac{\mathbb P(W=w,X=x,Z=z)}{\mathbb P(W=w)}=\frac{\mathbb P(W=w,X=x)}{\mathbb P(W=w)}\frac{\mathbb P(W=w,Z=z)}{\mathbb P(W=w)}.\qquad (1) $$

Using this equation, I will show that your last equation (which you marked with an inequality) is in fact true, resolving the confusion. That is, I will show that the previous equation implies $$ \mathbb P(X\mid W)=\mathbb P(X\mid Z,W) $$ or equivalently in long form $$ \frac{\mathbb P(W=w,X=x)}{\mathbb P(W=w)}=\frac{\mathbb P(W=w,X=x,Z=z)}{\mathbb P(W=w,Z=z)}.\qquad (2) $$

So how do we show that $(1)$ implies $(2)$? Well, remember that on the right hand side of $(2)$, $w$ and $z$ take values such that $\mathbb P(W=w,Z=z)>0$. Thus, we can apply $(1)$ to $w,z$ with this property, in which case it is legal to divide through on both sides by $\mathbb P(W=w,Z=z)$, and this takes us from $(1)$ to $(2)$, after cancelling out the repeated factor of $\mathbb P(W=x)$ from both sides.