Conditional Independence vs Independence of Conditional Expectations

833 Views Asked by At

Are the two concepts distinct (is one stronger than the other) or equivalent?

In other words given two random variables $X$ and $Y$ defined on a sigma algebra $\mathcal{F}$ with sub sigma algebra $\mathcal{G} \subset \mathcal{F}$, is $E[X|\mathcal{G}] \perp E[Y|\mathcal{G}]$ (independence of the conditional expectations) equivalent to $X$ and $Y$ being conditionally independent given $\mathcal{G}$ (for an example, let $\mathcal{G} = \sigma(Z)$ for some random variable $Z$)?

What if $X$ and $Y$ were independent? Would that guarantee conditional independence (no!, I have counterexamples $X$ first coin flip, $Y$ second coin flip, $Z = X+Y$) or independence of conditional expectations?

1

There are 1 best solutions below

3
On

Consider the following very simple example.

Let $\pi$ $\sim U[0,1]$ or something like that -- a random number between $0$ and $1$. And let $$X=\begin{cases}1&\text{ with probability }\pi\\0&\text{ with probability }1-\pi&\end{cases}$$ and let $$Y=\begin{cases}1&\text{ with probability }1-\pi\\0&\text{ with probability }\pi\end{cases}$$ and make sure that if $\pi$ is given then $X$ and $Y$ realize independently.

That is $$P(X=1\mid \pi\ )=P(Y=0\mid\pi\ )=\pi,\ P(Y=1\mid \pi\ )=P(X=0\mid \pi\ )=1-\pi$$

and $X$ and $Y$ are conditionally independent with respect to $\pi$:

$$P(X=1\cap Y=1\mid \pi\ )=P(X=0\cap Y=0\mid \pi\ )=\pi(1-\pi)$$

$$P(X=0\cap Y=1\mid \pi\ )=(1-\pi)^2$$ $$P(X=1\cap Y=0\mid \pi\ )=\pi^2.$$


Now, compute the conditional expectations:

$$E[\ X\mid \pi\ ]=\pi, \ \text{ and } E[\ Y\mid \pi\ ]=1-\pi.$$

$\pi$ and $1-\pi$ are not independent.


EDIT

Considering independent $X$ and $Y$.

Let $\Omega=\{1,2,3,4,5,6,7,8,9\}$, $\mathscr F=2^{\Omega}$, $P(I)=\frac19$ and let's visualize $\Omega$ lie this:

Furthermore let $$X(\omega)=\begin{cases}1&\text{ if }\omega \in\{1,2,3\}\\ 2&\text{ if }\omega \in\{4,5,6\}\\ 3&\text{ if }\omega \in\{7,8,9\} \end{cases}$$ and let $$Y(\omega)=\begin{cases}1&\text{ if }\omega \in\{1,4,7\}\\ 2&\text{ if }\omega \in\{2,5,8\}\\ 3&\text{ if }\omega \in\{3,6,9\}. \end{cases}$$ Check that $X$ and $Y$ are independent.

Now, define $\mathscr G$ as $$\{\Omega,\emptyset, G_1=\{1,2,4,5,7\},G_2=\{3,6,8,9\}\}.$$

In order to get the conditional expectations $E[X\mid \mathscr G\ ]$ , we need to calculate the following conditional probabilities:

$$P(X=i\mid G_1\ )=\frac{P(X=i \cap G_1 )}{P(G_1)}=\begin{cases}\frac35&\text{ for }&i=1\\\frac25&\text{ for }&i=2\\0&\text{ for }& i=3,\end{cases}$$ $$P(X=i\mid G_2\ )=\frac{P(X=i \cap G_2 )}{P(G_2)}=\begin{cases}0&\text{ for }&i=1\\\frac14&\text{ for }&i=2\\\frac34&\text{ if }&i=3.\end{cases}$$ So

$$E[\ X\mid G_1\ ]= 1\times P(X=1\mid G_1\ )+2\times P(X=2\mid G_1)+3\times P(X=3\mid G_1\ )=$$ $$=\frac35+2\times \frac25=\frac75$$ and $$E[\ X\mid G_2\ ]= 1\times P(X=1\mid G_2\ )+2\times P(X=2\mid G_2)+3\times P(X=3\mid G_2\ )=$$ $$=2\times \frac14+3\times \frac34=\frac{11}4$$ then

$$E[X\mid \mathscr G\ ](\omega)=\begin{cases}\frac75&\text{ if }&\omega\in G_1\\\frac{11}4&\text{ if }&\omega\in G_2\end{cases}.$$

In order to get the conditional expectations $E[Y\mid \mathscr G\ ]$ , we need to calculate the following conditional probabilities:

$$P(Y=i\mid G_1\ )=\frac{P(Y=i \cap G_1 )}{P(G_1)}=\begin{cases}\frac25&\text{ for }&i=1\\\frac25&\text{ for }&i=2\\\frac15&\text{ for }& i=3,\end{cases}$$ $$P(Y=i\mid G_2\ )=\frac{P(Y=i \cap G_1 )}{P(G_2)}=\begin{cases}\frac14&\text{ for }&i=1\\\frac14&\text{ for }&i=2\\\frac12&\text{ if }&i=3.\end{cases}$$ So

$$E[\ Y\mid G_1\ ]= 1\times P(Y=1\mid G_1\ )+2\times P(Y=2\mid G_1)+3\times P(Y=3\mid G_1\ )=$$ $$=\frac25+2\times \frac25+3\times \frac15=\frac95$$ and $$E[\ Y\mid G_2\ ]= 1\times P(Y=1\mid G_2\ )+2\times P(Y=2\mid G_2)+3\times P(Y=3\mid G_2\ )=$$ $$=1\times \frac14+2\times \frac14+3\times \frac12=\frac{9}4$$ then

$$E[Y\mid \mathscr G\ ](\omega)=\begin{cases}\frac95&\text{ if }&\omega\in G_1\\\frac{9}4&\text{ if }&\omega\in G_2\end{cases}.$$


As far as the independence of $E[X\mid\mathscr G\ ]$ and $E[Y\mid\mathscr G\ ]$. Consider the following example

$$P\left(E[\ X\mid \mathscr G\ ]=\frac75\right)=P(G_1)=\frac59,\ \ P\left(E[\ Y\mid \mathscr G\ ]=\frac95\right)=P(G_1)=\frac59$$ and $$P\left(E[\ X\mid \mathscr G\ ]=\frac75 \cap E[\ Y\mid \mathscr G\ ]\right)=P(G_1)=\frac59$$ and not $\frac{25}{81}$.