Conditional Expectation of 3 random variables

Question

Can we find three random variables $X_1, X_2, X_3$ such that $E[X_2 \mid X_1] = X_1$ and $E[X_3 \mid X_2] = X_2$ but $E[X_3 \mid X_1, X_2] \neq X_2$?

I'm stuck on how to choose such random variables; in particular how to define $X_3$ in relation to the other two. At first, I was thinking of setting $X_n = \frac{S_n}{n}$ where $S_n = Y_1 + Y_2 + ... Y_n$ for an IID sequence of random variables $Y_n$, $n \geq 1$ and $S_0 = 0$. Then an easy choice for $X_2$ is $\frac{S_{n + 1}}{n + 1}$, as it can be shown that $E[Y_1 \mid S_n] = \frac{S_n}{n}$. But I am unsure how to make the expectation of $X_3$ conditioned on $X_1$ and $X_2$ have dependencies beyond $X_2$ only. Any help would be appreciated!

Answer 1

Let $X_1,X_0$ be iid Bernoulli fair variables: $P(X_i=0)=P(X_i=1)=1/2$

Let $X_2 = 2 X_1 X_0$.

We get $E[X_2| X_1]= 2 X_1 E[X_0] = 2 X_1 \frac12 = X_1$ and $E[X_2]= E[E[X_2| X_1]]= 1/2$. Also

$$E[X_1 | X_2 ] = \frac13 (X_2+1)$$

Now, let $X_3 = \frac23 X_2 + X_1 -\frac13$

We get

$$E[X_3|X_2] = \frac23 X_2 + \frac13 (X_2+1) -\frac13 = X_2$$

But

$$E[X_3|X_2,X_1] = \frac23 X_2 + X_1 -\frac13$$

Answer 2

Take any $X_1\neq X_2$ such that $\mathbb E[X_2\vert X_1]=X_1$ and set $X_3=X_1+X_2-\mathbb E[X_1\vert X_2]$.

Note that whatever example you take, you'll have $X_3\neq X_2$, see e.g. this question.

Then $\mathbb E[X_3\vert X_2]=X_2$ but $\mathbb E[X_3\vert X_1,X_2]=X_3\neq X_2$.