How to show $E(\hat{y}|y)=E(E(\hat{y}|\theta,y)|y)$?

Question

I am reading the following section of "bayesian data analysis" by Gelman. I do not quite understand why $E(\hat{y}|y)=E(E(\hat{y}|\theta,y)|y)$.

It would be nice if you can also explain the equation for the variance. Thanks a lot.

Answer 1

This is just the tower law for conditional expectation. If you understand why $E(E(Y\mid X)) = E(Y),$ think about doing this same computation, only instead of averaging over the whole probability space, doing it conditionally on a third random variable $Z.$ You'd get $$ E(E(Y\mid X,Z)\mid Z) = E(Y\mid Z).$$

Similarly, the variance equation is just the familiar law of total variance $$ Var(X) = E(Var(X\mid Y)) + Var(E(X\mid Y))$$ which will also go through when you condition on some random variable $Z$ at the outset. So it becomes $$ Var(X\mid Z) = E(Var(X\mid Y)\mid Z)+Var(E(X\mid Y)\mid Z)$$