Show that $\operatorname{Var}(Y −E(Y\mid X))=\operatorname{Var}(\operatorname{Var}(Y\mid X))$.

Question

I tried to use the identity that $\operatorname{Var}(Y)+\operatorname{Var}(E(Y\mid X) - 2\operatorname{Cov}(Y, E(Y\mid X))$ as a starting point however, I'm not so sure where to go from there. The formula of covariance is a fraction, which seems like it will screw up the whole thing, so is there another way to do this?

Answer 1

If X and Y are random variables defined on the same probability space, then the conditional variance of X given that Y = y is $Var(X|Y=y) = E((X - E(X|Y=y))^2|Y=y)$. The same formula for random variable Y is $Var(X|Y) = E((X - E(X|Y))^2|Y)$. Taking expectation on both sides, we get $E(Var(X|Y)) = E(E((X - E(X|Y))^2|Y)) = E((X-E(X|Y))^2) = Var(X-E(X|Y))$.

Answer 2

The formula for covariance is not a fraction. (Are you perhaps thinking of correlation coefficient?)

It is $\mathsf {Cov}[Y, \mathsf E[Y\mid X]] = \mathsf E[Y\mathsf E[Y\mid X]]-\mathsf E[Y]\mathsf E[\mathsf E[Y\mid X]]$

Which you can simplify down a lot, using the law of iterated expectation.

$\mathsf E[\mathsf E[Y\mid X]] = \mathsf E[Y] \\ \mathsf E[Y\mathsf E[Y\mid X]] = \mathsf E[Y]^2$

Answer 3

Decompose $Y$ into two orthogonal parts: $Y=E(Y|X)+(Y-E(Y|X))$ so that $$Var(Y)=Var(E(Y|X))+Var(Y-E(Y|X))$$ Use $$Var(Y)=Var(E(Y|X))+E(Var(Y|X))$$ to complete the proof.

ETA: It could be easier to see this by simply writing down definition of conditional variance $$Var(Y|X)=E((Y-E(Y|X))^2|X)$$ and taking expectation of both sides, noting that $E(Y-E(Y|X))=0$.