Show that $\mathbb{E}[h(x)^{2}-2h(x)y+y^{2}]=\mathbb{E}[h(x)^{2}-2h(x)\mathbb{E}[y|x]+\mathbb{E}[y^{2}|x]]$

Question

Let $\mathcal{D}$ be a distribution over $Z=X\times Y$. I am trying to understand the following:

Why $\mathbb{E}_{(x,y)\sim\mathcal{D}}[h(x)^{2}-2h(x)y+y^{2}]=\mathbb{E}_{x\sim\mathcal{D}_{x}}[h(x)^{2}-2h(x)\mathbb{E}[y|x]+\mathbb{E}[y^{2}|x]]$ s.t $\mathcal{D}_{x}$ is the marginal distribution over x?

What is the connection between the destribution $\mathcal{D}$ and the margial distribution $\mathcal{D}_x$? How can we just "switch" between distributions?

Answer 1

We have \begin{align} &\mathsf{E}_{(x,y)\sim \mathcal{D}}[h(x)^2-2h(x)y+y^2] \\ &=\mathsf{E}_{(x)\sim \mathcal{D}_x}\mathsf{E}_{y\lvert x \sim \mathcal{D}_{y|x}}[h(x)^2-2h(x)y+y^2] \\ &=\mathsf{E}_{(x)\sim \mathcal{D}_x} \left(\mathsf{E}_{y\lvert x \sim \mathcal{D}_{y|x}}(h(x)^2-2h(x)\mathsf{E}_{y\lvert x \sim \mathcal{D}_{y|x}}(y)+\mathsf{E}_{y\lvert x \sim \mathcal{D}_{y|x}}(y^2)]\right) \\ &=\mathsf{E}_{(x)\sim \mathcal{D}_x} \left(h(x)^2-2h(x)\mathsf{E}(y\lvert x) +\mathsf{E}(y^2\lvert x)\right) \\ \end{align}