Why is Borel measurability needed here?

Question

Let $X,Y$ be random variables on the probability space $(\Omega,\mathcal{F})$. Since $P(X|Y)$ is $\sigma(Y)$-measurable, we have by the Doob-Dynkin lemma that $P(X|Y)=h(Y)$ for some Borel-measurable function $h$. Now we define $P(X|Y=y):=h(y)$. Suppose that there is some other $\alpha$, such that $P(X|Y=y)=\alpha(y)$, $PX^{-1}$-a.e $y\in\mathbb{R}$. Then if $\alpha$ is Borel measurable, $P(X|Y)=\alpha(Y)$ $P$-a.e. $\omega\in\Omega$.

Why is the condition $\alpha$ is borel-measurable needed?

Answer 1

It seems that you may have a few problems understanding the meaning of conditional expectation, which is worsen by the sloppy notation $\mathbb{E}[X|Y=y]$ for $\mathbb{E}[X|\sigma(Y)]$.

For the rest of this posting we will use the following notation

1. $ f:(E,\mathscr{E})\rightarrow(R,\mathscr{R})$ to denote that $(E,\mathscr{E})$ and $(R,\mathscr{R})$ are measurable spaces; $f$ is a function with domain $E$ taking values in $R$, and that $f^{-1}(F)\in\mathscr{E}$ for any $F\in\mathscr{R})$. This is the same as to say that $f$ is an $\mathscr{E}/\mathscr{R}$-measurable function.
2. When $R$ is the real line $\mathbb{R}$ and $\mathscr{R}$ is the Borel $\sigma$-algebra on $\mathscr{R}$, then simply say that $f$ is a real valued $\mathscr{E}$-measurable function.

Regardless of how this concept has been presented to you (whether through orthogonal projections of $L_2$ functions, or through Radon-Nykodim's theorem) notice that all the machinery behind conditional expectation relies on integration, and so all functions and sets involved in the development of the concept of conditional expectation are measurable.

Recall that

Definition. Given a probability space $(\Omega,\mathscr{F},\mathbb{P})$, and integrable function $X:(\Omega,\mathscr{F})\rightarrow(\mathbb{R},\mathscr{B}(\mathbb{R}))$, and a sub $\sigma$-albegra $\mathscr{A}$, i.e., a $\sigma$-algebra $\mathscr{A}\subset\mathscr{F}$, the conditional expectation of $X$ given $\mathscr{A}$ is an $\mathscr{A}$- measurable function $Y:(\Omega,\mathscr{A})\rightarrow(\mathbb{R},\mathscr{B}(\mathbb{R}))$ such that \begin{align} \mathbb{E}[X\mathbb{1}_A]=\int\mathbb{1}_AX\,d\mathbb{P}=\int\mathbb{1}_AY\,d\mathbb{P}\tag{1}\label{one} \end{align} for all $A\in\mathscr{A}$.

Observations:

• Integration theory implies that there exists an $\mathscr{A}$-measurable function $Y$ that satisfies \eqref{one}
• Any two $\mathscr{A}$-measurable functions $Y$ and $Y'$ that satisfy \eqref{one} are equal $\mathbb{P}$-almost surely, i.e. $\mathbb{P}[\{Y\neq Y'\}]=0$.

So, the conditional expectation of $X$ given $\mathscr{A}$ exists and is unique $\mathbb{P}$-almost surely. Once existence and uniqueness is out of the way, it is safe to introduce some notation. Choose one $Y$ satisfying \eqref{one} and denote it as $\mathbb{E}[X|\mathscr{A}]$. Again, notice that $E[X|\mathscr{A}]$ is an $\mathscr{A}$-measurable function.

Now, suppose $Y:(\Omega,\mathscr{F})\rightarrow(E,\mathscr{E})$. Notice that $\sigma(Y)=\{Y^{-1}(F):F\in \mathscr{E}\}$, the $\sigma$-algebra generated by $Y$, is a sub-$\sigma$ algebra of $\mathscr{F}$. Then the conditional expectation of $X$ given $\sigma(Y)$, i.e. $\mathbb{E}[X|\sigma(Y)]$, is usually denoted by $\mathbb{E}[X|Y]$, that is $\mathbb{E}[X|Y]:=\mathbb{E}[X|\sigma(Y)]$.

• Recall that (Doob-Dynkin) that if $\Phi$ is a real-valued $\sigma(Y)$-measurable function, i.e., $\Phi:(\Omega,\sigma(Y))\rightarrow(\mathbb{R},\mathscr{B}(\mathbb{R}))$, then there exists a real valued $\mathscr{E}$-measurable function $h:(E,\mathscr{E})\rightarrow(\mathbb{R},\mathscr{B}(\mathbb{R})$ such that $$ \Phi=h\circ Y $$

Applying this to $\mathbb{E}[X|\sigma(Y)]$ yields a function real valued $\mathscr{E}$-measurable function such that $$\mathbb{E}[X|\sigma(Y)]=h\circ Y$$ Equality here is $\mathbb{P}$-almost surely.

In this context, a few words about the unfortunate (although some time useful) notation $E[X|Y=y]$ are in order. Given $y\in E$, $\{Y=y\}:=\{\omega\in\Omega: Y(\omega)=y\}\in\mathscr{F}$. On this set, $\mathbb{E}[X|\sigma(Y)]=h(y)\quad$ $\mathbb{P}$-almost surely, that is the set $$\{\omega\in\Omega: Y(\omega)=y\quad\text{and}\quad \mathbb{E}[X|\sigma(Y)](\omega)\neq h(y)\}$$ has $\mathbb{P}$-measure $0$. This fact is summarized by the notation $$\mathbb{E}[X|Y=y]=h(y)$$ One should not, however, confuse $\mathbb{E}[X|Y=y]$ with $E[X|\sigma(Y)]$, as the former is not a random variable but a notation, and the later is a random variable.

I hope this clarifies some aspects of your understanding of conditional expectation and answers your question.