Showing that we can always decompose $Y = \mathbb{E}\left[Y|X\right] + \epsilon$.

Question

The question is that

Let $Y$ be a random variable with finite mean and $X$ a random vector. $Y$ can always be decomposed as follows:

$Y = \mathbb{E}\left[Y|X\right] +\epsilon$,

where $\epsilon$ is a scalar random variable that is uncorrelated with any function of $X$. True or False?

Since there is no limitation on what sort of function it should be I just defined the function $f:H_1 \rightarrow H_2$, where $H_1$ is the space of all real random vectors with dimension $\dim X$ and $H_2$ is the space of all real random variables, and let $f$ map any $A\in H_1$ to $Y - \mathbb{E}\left[Y|A\right]$. Then $\epsilon$ should correlate with $f(X)$. I don't see why this example doesn't work because I'm pretty sure the answer should be True.

The proof that it should be true looks something like:

$\mathbb{E}\left[\epsilon f(X)\right] = \mathbb{E}\left[\mathbb{E}\left[\epsilon f(X)\;|X\right]\right] = \mathbb{E}\left[f(X)\mathbb{E}\left[\epsilon\;|X\right]\right] = 0$, since

$\mathbb{E}\left[\epsilon\;|X\right] = \mathbb{E}\left[Y\;|X\right] - \mathbb{E}\left[\mathbb{E}\left[Y\;|X\right]\;|X\right] = \mathbb{E}\left[Y\;|X\right] - \mathbb{E}\left[Y\;|X\right] = 0$, and

$\mathbb{E}\left[\epsilon\right] = 0$, so that $\mathrm{Cov}\left(\epsilon, f(X)\right) = 0$.

Answer 1

Just to clarify some of the steps since the phrase "by definition of conditional expectation" is not quite accurate; instead, one is using linearity, measurability, and the tower property. :)

First, for any $\sigma(X)$-measurable $U$ we have \begin{eqnarray*} {\bf E}[\epsilon U] &=& {\bf E}[(Y-{\bf E}[Y|X])U] \qquad\text{(by definition of }\epsilon \text{)}\\ &=& {\bf E}[UY]-{\bf E}[{\bf E}[Y|X]U] \qquad\text{(by linearity of expectation)}\\ &=& {\bf E}[UY]- {\bf E}[{\bf E}[UY|X]] \qquad \text{(}U \text{ is }\sigma(X)\text{-measurable, and property of conditional expectations)}\\ &=& {\bf E}[UY]- {\bf E}[UY] \qquad \text{(by tower property of conditional expectations)}\\ &=& 0 \end{eqnarray*} And, second, \begin{eqnarray*} {\bf E}[\epsilon] &=& {\bf E}[Y-{\bf E}[Y|X]] \qquad\text{(by definition of }\epsilon \text{)}\\ &=& {\bf E}[Y]-{\bf E}[{\bf E}[Y|X]] \qquad\text{(by linearity of expectation)}\\ &=& {\bf E}[Y]- {\bf E}[Y] \qquad \text{(by tower property of conditional expectation)}\\ &=& 0 \end{eqnarray*} Hence, \begin{eqnarray*} {\bf Cov}[\epsilon,U] &=& {\bf E}[(\epsilon -{\bf E}[\epsilon])(U-{\bf E}[U])] \qquad\text{ (by definition of covariance)}\\ &=& {\bf E}[\epsilon U]-{\bf E}[\epsilon]{\bf E}[U] \qquad\text{ (using linearity of expectation)}\\ &=& 0 - 0\times {\bf E}[U] \qquad\text{ (using above results)}\\ &=& 0 \end{eqnarray*} Using $\sigma(X)$-measurable $U$ in the above includes the case of $\mathbb{R}$-valued measurable functions $g$ of $X$ where one uses the usual interpretation of $g(X)$ as the random variable with value $g(X(\omega))$ at a particular $\omega$.

Answer 2

If you write $\varepsilon=Y-\mathbb E[Y|X]$, and compute for any $X$-measurable random variable $U$ the quantity \begin{align*} \mathbb E[\varepsilon U] &= \mathbb E[(Y-\mathbb E[Y|X]) U]\\ &=\mathbb E[YU]-\mathbb E[\mathbb E[Y|X] U]\\ &=\mathbb E[YU]-\mathbb E[YU]\\ &= 0 \end{align*} by definition of conditional expectation.

Your example is not well defined, "$f$ maps any $A$ to $Y-\mathbb E[Y|A]$", the second term is still random, it is not a value.