Using Adam's Law (Law of Total Expectation) to find expectation of residual

Question

This may seem like a rather simple question, but I haven't been able to come up with an explanation myself or find one on the Internet.

I've learned that Adam's Law states that

$$E(E(Y|X)) = E(Y)$$

While solving an exercise problem, I noticed that the solution used the fact that

$$E(E(Y|X)|X) = E(Y|X)$$

I'm a bit confused as to how this equality has been derived. The form of Adam's Law with extra conditioning I'm familiar with is

$$E(E(Y|X, Z)|Z) = E(Y|Z)$$

but there seems to be something missing in the equation that I've given.

Would anybody be able to help me understand the derivation?

Thank you.

EDIT

The textbook that I'm using is Introduction to Probability (1e) - Blitzstein & Hwang.

There isn't particularly a specific definition regarding the property that I mentioned. If I were to quote the textbook exactly:

Conditional expectation has some very useful properties, that often allow us to solve problems without having to go all the way back to the definition.

...

Theorem 9.3.7 (Adam's Law): For any random variables $X$ and $Y$,

$$E(E(Y|X)) = E(Y)$$

Adam's law with extra conditioning:

$$E(E(Y|X, Z) = E(Y|Z)$$

The remainder of the textbook related to the law is about how to prove the equality, which I understand how to do.

Here's the specific exercise problem that has led me to ask this question:

Let $X$ and $Y$ be random variables with finite variances, and let $W = Y - E(Y|X)$.

Compute $E(W|X)$.

My approach is as follows:

\begin{align} E(W|X) & = E(Y - E(Y|X) | X) \\ & = E(Y|X) - E(E(Y|X)|X) \\ \end{align}

and this is the particular part that threw me off. The correct answer would be that $E(E(Y|X)|X) = E(Y|X)$ therefore giving us $0$.

Answer 1

All you need it to read your book (the one by Blitzstein-Hwang you mentioned in the post) more thoroughly, really.

The definition you need is that of conditional expectations given a random variable. The first thing you should know is what the notation $E(Y|X)$ means before discussing anything about it. Section 9.2: Conditional expectation given an r.v. is the place you should refer to in the first place.

Note that the whole Section 9.3 is about "Properties of conditional expectation". In particular, you have

Applying Theorem 9.3.2, one has $$ E(E(Y|X)|X)=E(g(X)\cdot 1|X)=g(X)E(1|X)=g(X)\cdot 1=g(X), $$ where $g(X):=E(Y|X)$.

---

[Added:] Note that a rigorous measure theory based answer to your question would be very different. (Even Definition 9.2.1 and statement of Theorem 9.3.2 would change in that setting.)

Answer 2

As commenters point out, a definition would clear things up. Rick Durrett's Probability (free online) presents $E(X|\mathcal F)$ as the random variable $Z$ so that

1) $Z\in\mathcal F$, and

2) $\int_A XdP=\int_AZdP$ for all $A\in\mathcal F$.

To prove your fact, we have to show that

1) $E(E(Y|X)|X)\in \sigma(X)$, and

2) $\int_AE(E(Y|X)|X)dP=\int_AYdP$.

The first part follows from the fact that $E(E(Y|X)|X)$ is a random variable that results from conditioning on $\sigma(X)$. ($E(Y|X)$ means exactly $E(Y|\sigma(X))$.)

The second part follows from repeatedly applying property 2) for conditional expected values: For all $A\in\sigma(X)$,

$$\int_AE(E(Y|X)|X)dP=\int_AE(Y|X)dP=\int_AYdP,$$ with first equality by applying property 2) to the random variable that results from taking the expectation of $E(Y|X)$, conditioning on $\sigma(X)$; the second equality follows from applying property 2) to the random variable that results from taking the expectation of $Y$, conditioning on $\sigma(X)$.

Answer 3

Let's use Rick Durrett's definition of conditional expectation to prove this result. Given a probability space $(\Omega,\mathcal{F}_0,P)$, and a $\sigma$-field $\mathcal{F} \subset \mathcal{F}_0$, and a random variable $X \in \mathcal{F}_0$ with $E[|X|]<\infty$, we define the conditional expectation of of $X$ given $\mathcal{F}$, $E[X|\mathcal{F}]$, to be a random variable $Y$ that has the following properties

1. $Y$ is $\mathcal{F}$-measurable.
2. for all $A \in \mathcal{F}, \, \int_A Y \, dP=\int_A X \, dP$.

We also know that conditional expectation exists and is unique (Chapter 5.1 - Durrett - Probability Theory and Examples - 4th Edition), so if $Y$ and $Y'$ satisfy conditions 1. and 2. then $Y\overset{a.s.}{=}Y'$.

Now on to the matter of the law of total expectation.

First note that since $\mathcal{F}$ is a $\sigma$-algebra, $\Omega \in \mathcal{F}$. Next, since $E[X|\mathcal{F}]$ is a random variable such that for all $ A \in \mathcal{F}, \, \int_A E[X|\mathcal{F}] \, dP = \int_A X \, dP$, this must also be true for $A=\Omega$.

Therefore we get

$$E[E[X|\mathcal{F}]] = \int_\Omega E[X|\mathcal{F}] \, dP = \int_\Omega X \, dP = E[X].$$