What does $\mathbb{E}[X|Y]$ (conditional expectation) mean?

Question

I am wondering what the conditional expectation $\mathbb{E}[X|Y]$ means- when I looked it up on Wikipedia, the only definition given is for $\mathbb{E}[X|Y=y]$ for some value $y$ which $Y$ assumes. I am mainly wondering what it means to condition on an entire random variable rather than an event.

Answer 1

You can view $E[X \mid Y=y]$ as a function $f(y) := E[X \mid Y=y]$.

Then $E[X \mid Y]$ is defined as $f(Y)$. This is actually mentioned at the very top of the Wikipedia page in the second paragraph:

Answer 2

An example may be illustrative. Consider a fair six-sided die (numbered from $1$ to $6$ inclusive). Let $X \in \{1,2,3,4,5,6\}$ be the outcome of a single die roll. Then define $$Y = \begin{cases} 1, & X \text{ is prime} \\ 0, & X \text{ is not prime }. \end{cases}$$ That is to say, if you roll a $2$, $3$, or $5$, then $Y = 1$, and if you roll a $1$, $4$, or $6$, then $Y = 0$. (Note that $1$ is not prime.)

Now if I ask you, what is $\mathbb E[X]$, what does this represent? It is the expected value of the outcomes of a single die roll. This is just $$\mathbb E[X] = \frac{1+2+3+4+5+6}{6} = \frac{7}{2} = 3.5.$$ It represents the "theoretical average," a value that we would get close to seeing if we took the arithmetic mean of the outcomes of many die rolls.

But if I ask you, what is $\mathbb E[X \mid Y = 1]$, how would you interpret this? It is simply the same kind of average as before, except we have restricted ourselves to counting only those outcomes for which the die roll yielded a prime number. That is to say, $$\mathbb E[X \mid Y = 1] = \frac{2 + 3 + 5}{3} = \frac{10}{3}.$$ Why did we divide by $3$ instead of $6$? The reason is because the conditional probability of any one of the three outcomes $\{2,3,5\}$, given that we rolled one of these numbers, is $1/3$, because each of these outcomes remains equiprobable: given you rolled a prime number, the chance it was $2$ is the same as the chance it was $3$, which is the same as the chance it was $5$.

Similarly, we have $$\mathbb E[X \mid Y = 0] = \frac{1+4+6}{3} = \frac{11}{3}.$$ Notice that this conditional expectation is not equal to $\mathbb E[X \mid Y = 1]$; moreover, neither is equal to $\mathbb E[X]$. What this shows us is that $\mathbb E[X \mid Y = y]$ depends on whether $Y = 0$ or $Y = 1$. It is a function of $y$. We could write it as

$$\mathbb E[X \mid Y = y] = \begin{cases} \frac{10}{3}, & y = 0 \\ \frac{11}{3}, & y = 1. \end{cases}$$

But we can be a little clever and also write it as $$\mathbb E[X \mid Y = y] = \frac{10 + y}{3},$$

without relying on piecewise function notation, because $Y \in \{0, 1\}$, and $11 = 10+1$. We can also omit the $y$:

$$\mathbb E[X \mid Y] = \frac{10 + Y}{3}.$$

What is the distinction between this version and the previous? This one emphasizes that $\mathbb E[X \mid Y]$ is a function of a random variable, in this case, $Y$. As such, it is also itself a random variable. The previous version is just saying "this is a function of some variable $y$, which does not necessarily need to be random."

---

Exercise 1. Given $X$ and $Y$ defined as above, what is $\operatorname{Var}[X \mid Y]$?

Exercise 2. What is $\mathbb E[\operatorname{Var}[X \mid Y]]$? Note that the expectation is with respect to $Y$, so your answer should be a number.

Exercise 3. Suppose $$Z = \begin{cases} 1, & X \text{ is even} \\ 0, & X \text{ is odd}. \end{cases}$$ What is $\mathbb E[X \mid (Y = 1) \cap (Z = 1)]$? What is $\mathbb E[Z \mid Y]$?