Prove that $U=Y - E[Y|X]$ and $X$ are uncorrelated

Question

Let $U = Y - E[Y|X]$. How can I prove that $U$ and $X$ are not correlated?

I've been doing a lot of things but when I calculate $\text{cov}(U,X)$ I finish with $EXY - EXEY$ and not $0$ which would be the result.

Any help, guys?

Thanks

Answer 1

Hints:

1. Verify that $E(X Y \mid X) = X E(Y \mid X)$.
2. Use the tower property of conditional expectation to show that $$ E(Y) = E \big[ E(Y \mid X) \big].$$
3. Conclude from Step 2 that $E(U)=0$.
4. Use Step 1+2 and the linarity of the expectation to prove that $$E(UX) = 0.$$
5. Combining Step 3 and 4 yields $$E(UX) = E(U) E(X),$$ i.e. $U$ and $X$ are uncorrelated.

Remark: You will need some integrability conditions on $X$ and $Y$ to ensure that the (conditional) expectations and the covariance are well-defined.

Answer 2

I will give you the idea and intuition and you'll be able to get the rest easily.

As known, $ \mathbb{E} \left[ Y \mid X \right] $ is the best estimator of $ Y $ given $ X $ in the MMSE sense.

Since it is the optimal estimator in MMSE sense it obeys the Orthogonality Principle.

Just derive those and you'll get the answer as the term in your question is the estimation error which, according to the Orthognality Principle, is un correlated to the data.

Answer 3

$$E[XU] = E[X(Y - E[Y|X])] = E[XY - XE[Y|X]] = E[XY] - E[XE[Y|X]]$$

Because $X$ is a function of $X$, we can pull out $X$: $XE[Y|X] = E[XY|X]$. Then $E[XE[Y|X]] = E[E[XY|X]]$, so

$$E[XU] = E[XY] - E[E[XY|X]]$$

Then $E[E[XY|X]] = E[XY]$. You can say this follows from tower property, but you can just say this is total expectation. Finally

$$E[XU] = E[XY] - E[XY] = 0$$