Let $\Omega= [0,1]$, $P$ be Lebesgue measure. Let $$Y(x) = \begin{cases} x^2, & \mbox{if } x \in [0, \frac{1}{3}) \\ \frac{1}{9}, & \mbox{if } x \in [\frac{1}{3}, \frac{2}{3}) \\ (x-1)^2, & \mbox{if } x \in [\frac{2}{3}, 1 ) \end{cases}$$
and $X(x)=x$.
I am supposed to find $\mathbb{E}(X|Y)$ and $\mathbb{E}(Y|X)$.
The second one is easy, because $\sigma(X) = \mathcal{B}([0,1])$ (borel sets), so $Y$ is measurable with respect to $\sigma(X)$ and so $\mathbb{E}(Y|X)$ = Y.
But the first one is more troublesome.
I predict that $$\mathbb{E}(X|Y)(x) = \begin{cases} ? & \mbox{if } x \in [0, \frac{1}{3}) \\ a & \mbox{if } x \in [\frac{1}{3}, \frac{2}{3}) \\ ? & \mbox{if } x \in [\frac{2}{3}, 1 ) \end{cases}$$ and I computed that $$\int_{1/3}^{2/3} a dx = \int_{1/3}^{2/3} x dx = \frac{1}{6} = \frac{1}{3}a \implies a=\frac{1}{2}.$$
But I don't know what to do with $x^2$ and $(x-1)^2$ part.
Could you help me with that?
I'm assuming
$\mathscr F = \mathscr B([0,1])$ or $\mathscr F = \mathscr M([0,1])$
$X$ and $Y$ are $(\mathbb R, \mathscr B(\mathbb R))$-valued.
If you graph $Y$, you will see that the range of $Y$ is $[0,1/9]$
$$E[X|Y] = E[X|Y \in [0,1/9)]1_{Y \in [0,1/9)}(\omega) + E[X|Y \in \{1/9\}]1_{Y \in \{1/9\}}(\omega)$$
$$:= E[X|Y \in A]1_{Y \in A}(\omega) + E[X|Y \in A^C]1_{Y \in A^C}(\omega)$$
Now $Y \in A \iff X \in [0,1/3) \cup (2/3,1] := D \iff \omega \in D$
Hence
$$1_{Y \in A}(\omega) = 1_{X \in D}(\omega) = 1_D(\omega)$$
$$E[X|Y \in A] = E[X|X \in D] = E[X|D]$$
$$E[X|Y \in A^C] = E[X|X \in D^C] = E[X|D^C]$$
$$1_{Y \in A^C}(\omega) = 1_{X \in D^C}(\omega) = 1_{D^C}(\omega)$$
Now
$$E[X|D] = E[X 1_D]/P(D)$$
where $P(D) = 2/3$ and
$$E[X 1_D] = \int_{\Omega} X 1_D dP = \int_{\mathbb R} t 1_D d\mathcal L_X(t)$$
$$= \int_{\mathbb R} t 1_D dF_X(t) \tag{*}$$
$$= \int_{D} t dF_X(t)$$
$$= \int_0^{1/3} t dF_X(t) + \int_{2/3}^1 t dF_X(t)$$
$$= \int_0^{1/3} t f_X(t) dt + \int_{2/3}^1 t f_X(t) dt$$
$$= \int_0^{1/3} t dt + \int_{2/3}^1 t dt$$
$$= (t^2/2)|_0^{1/3} + (t^2/2)|_{2/3}^1$$
$$= ((1/3)^2/2) + (1^2/2) - ((2/3)^2/2) = 1/3$$
Case is /// for $E[X|D^C] = 1/6$. I think you can check your answer by checking that:
$$1/2 = E[X] = E[E[X|Y]] = E[X1_D] + E[X1_{D^C}]$$
$(*)$
$$F_X(t) = P(X \le t) = P(X \in [-\infty,t] \cap \Omega) = P(X \in [0,t]) = P(\omega \in [0,t]) = P([0,t]) = \lambda([0,t]) = t$$
$$\to dF_X(t) = f_X(t) dt = 1 dt = dt$$