Today in probability class we received this exercise:
Let $\xi$ and $\eta$ be two random variables defined on a probability space $(\Omega,F, \mathbb{P})$. Show that
$inf_{f}\mathbb{E}(\eta - f(\xi))^2 = \mathbb{E}(\eta - \mathbb{E}(\eta|\xi))^2$
where infimum is taken over all Borel functions $f:\mathbb{R}\rightarrow \mathbb{R}$.
Maybe somebody has any ideas or hints on what I need to do here? Maybe I need to show two inequalities: than infimum is less or equal to $\mathbb{E}(\eta - \mathbb{E}(\eta|\xi))^2$ and, on the other hand, it's bigger than $\mathbb{E}(\eta - \mathbb{E}(\eta|\xi))^2$? From these two inequalities, I could state that equality holds.
I assume $\xi, \eta \in L^2$.
By the statement proved in this post we get that the set $L (\xi) := \{ \zeta \in L^2 (\Omega , F , \Bbb P) : \zeta \text{ is measurable w.r.t. } \sigma (\xi)\}$ is equal to $\{f (\xi) : f:\Bbb R \to \Bbb R \text{ measurable} , f(\xi ) \in L^2 \}$.
Since by this post the conditional expectation w.r.t. a $\sigma$-Algebra $\mathcal G \subset F$ is the orthogonal projection from $L^2 (\Omega , F, \Bbb P)$ onto $L^2 (\Omega , \mathcal G , \Bbb P)$ we have that, as orthogonal projection, for a random variable $X \in L^2 (\Omega, F , \Bbb P)$ it minimizes the distance between $X$ and arbitrary random variables in $L^2 (\Omega , \mathcal G , \Bbb P)$.
Your desired statement, thus follows by setting $\mathcal F = \sigma (\xi)$.
It can be also done by calculation, as can be seen here.