Law of iterated expectations

Question

I am studying a PhD in economics and I have some trouble with the understanding of the law of iterated expectations

In my test book, the definition is as follows:

Suppose that w is a random vector and y is a random variable. Let x be a random vector that is some function of w, say x = f(w). The most general statement of Law of Iterated Expectations that we will need are the following:

E(y|x) = E[E(y|w)|x] and E(y|x) = E[E(y|x)|w]

I do not understand two things:

1) The intuition behind the Law of iterated expectations

2) Why these two statements are equal.

If you can help me, I will appreciate a lot.

Answer 1

The intuition is based on the Law of Total Probability, and the definition of conditional probability.

When dealing with discrete random variables we have:

$$\mathsf E(Y\mid X{=}x) ~{= \sum_{y} y\,\mathsf P(Y{=}y\mid X{=}x) \\ = \sum_{(w,y)} y\,\mathsf P(W{=}y, W{=}w\mid X{=}x)\\ = \sum_{w}\sum_{y} y\,\mathsf P(Y{=}y\mid X{=}x,W{=}w)\;\mathsf P(W{=}w \mid X{=}x)\\ = \sum_{w}\mathsf E(Y\mid X{=}x,W{=}w)\,\mathsf P(W{=}w\mid X{=}x)\\ = \mathsf E(\mathsf E(Y\mid X{=}x,W{=}x)\mid X{=}x)}$$

When the random variables are continuous, with probability density functions, the integration expressions follow an analogous pattern, and this is extended to the sigma-algebra notation expressions of measure theory.

Thus $$\mathsf E(Y\mid X)=\mathsf E(\mathsf E(Y\mid X,W)\mid X)$$.   The right hand side may sometimes also be written as $\mathsf E(\mathsf E(Y\mid W)\mid X)$ .

---

However, likewise, $\mathsf E(\mathsf E(Y\mid X)\mid W)=\mathsf E(Y\mid W)$ and generally $\mathsf E(Y\mid W)$ and $\mathsf E(Y\mid X)$ are not the same thing.

Answer 2

1. Think of the conditional expectation $$\mathbb{E}[Y|\mathcal{G}]$$ as an approximation of $Y$ by averages based on the information given by the $\sigma$-algebra $\mathcal{G}$.
2. Think that the information provided by a random variable $W$ is coded in its $\sigma$-algebra $\sigma(W)$.
3. Since that $$\sigma(f(W)) \subset \sigma(W),$$ the iterated approximations $$ \mathbb{E}\left[\mathbb{E}[Y|W]|f(W)\right], ~\mathbb{E}\left[\mathbb{E}[Y|f(W)]|W\right]$$ will be reduced to $$\mathbb{E}[Y|f(W)].$$