Let $X$ and $Y$ be two random variable. Suppose $E[X|Y]=0$, we have $$E[XY]= E[\, E[ XY |Y ]\, ] = E[\, Y E[ X |Y ]\, ]= E[0] = 0$$ A geometric intuition also corroborates this fact if we think $E[X|Y]$ and $E[XY]$ as being the projection of $X$ onto $Y$ and the inner product, respectively. That is, the equation above tells me that if the projection of $Y$ on $Y$ is zero, then $X$ and $Y$ are orthogonal.
So I want to show the reciprocal: if $X$ and $Y$ are orthogonal, i.e., $E[XY]=0$, then the projection of $X$ on $Y$ is zero, i.e., $E[X|Y]=0$.
I've been trying to show this, but I couldn't.
- Is there any counterexample?
- If there is a counterexample, why does it go against geometric intuition?
The second question intrigues me more.
Counter-example: Let $Y\sim N(0,1)$ and $X=Y^{2}$. Then $E[XY]=0$ and $E[X|Y]=Y^{2}$. [Converses of true statements are often false].
About your intuition failing I can say the following:
$E[Y|X]$ is not the projection of $Y$ onto the vector space spanned by $X$. It is the projection of $Y$ onto the space of $X-$measurable functions. This last space includes $X^{2}$, for example, even though $X^{2}$ is not the vector space spanned by $X$. @AndréGoulart