I'm trying to find an example of $E(E(Y|X_1)|X_2)\ne E(E(Y|X_2)|X_1)$
My thought was taking $Y$ and $X_2$ independent, so $E(Y|X_2)=E(Y)$, and $Y$ and $X_1$ dependent as well as $E(Y|X_1)$ and $X_2$.
Then, I get: $E(E(Y|X_1)|X_2)\ne E(EY|X_1)=EY$
Now, I tried with some uniform distibutions, but I'm stuck.
Any help is appreciated.
The thing to see is that $E$ (averaging, which is a form of integration) loses information. The ordering says what gets thrown away.
Here's an example. Try taking $X_1, X_2$ to be independent binary random variables -- two coins, why not, so we can write the possible outcomes as hh, ht, th, tt, all of which have probability $\frac{1}{4}$. Then let's set $Y$ dependent on both, e.g.
$Y$ = 4 if hh, 8 if ht, 12 if th, 16 if tt
Note that $E(Y) = 10$, i.e. it's just a number - this gives some insight into the 'destructive' nature of averaging.
$E(Y|X_1)$ = 6 if $X_1$=h, 14 if $X_1$=t
$E(Y|X_2)$ = 8 if $X_2$=h, 12 if $X_2$=t
These functions have no dependency on the other $X$. The structure of Y (which depended on both) has been lost. So
$E(E(Y|X_1)|X_2)$ = 6 if ht or hh, 14 if tt or th
$E(E(Y|X_2)|X_1)$ = 8 if hh or th, 12 if ht or tt