Measure Theoretic Understanding of $P(X=x|Y=y)$

Question

1. The conditional probability $P(A|B)$ is defined to be $P(AB)/P(B)$, if $P(B)>0$.
2. The conditional probability $P(A|\mathcal F)$ is defined to be $E(1_A|\mathcal F)$, if $\mathcal F$ is a sigma-field.
3. Let $X,Y$ be jointly continuous random variables with joint density. We know there is a formula to compute $$ P(X=x|Y=y), $$ but since $P(Y=y)=0$, how could we understand it in a measure theoretic way?

Answer 1

There is no need to assume $X$ and $Y$ are jointly continuous.

First, observe that if $A$ is any Borel set, then $\mathbb{P}\{X \in A \, \mid \, \sigma(Y)\}$ is a $\sigma(Y)$-measurable random variable. Therefore, I ask you to consider the following exercise: if $X_{1}$ is a $\sigma(Y)$-measurable random variable, then there exists a Borel measurable function $g : \mathbb{R} \to \mathbb{R}$ such that $X = g(Y)$. In other words, $X$ is completely determined by the value of $Y$. In our case, we learn that there is a function $g_{A}$ such that $$\mathbb{P}\{X \in A \, \mid \, \sigma(Y)\}(\omega) = g_{A}(Y(\omega)).$$

Define $\mathbb{P}\{X \in A \, \mid \, Y = y\} = g_{A}(y)$. Now observe that if $B$ is a Borel set, then \begin{align*} \mathbb{P}\{X \in A, Y \in B\} &= \int_{Y^{-1}(B)} \mathbb{P}\{X \in A \, \mid \, \sigma(Y)\}(\omega) \, \mathbb{P}(d \omega) \\ &= \int_{\Omega} g_{A}(Y(\omega)) \chi_{B}(Y(\omega)) \, \mathbb{P}(d \omega) \\ &= \int_{-\infty}^{\infty} g_{A}(y) \chi_{B}(y) \, \mathbb{P}_{Y}(dy) \\ &= \int_{B} \mathbb{P}\{X \in A \, \mid \, Y = y\} \, \mathbb{P}_{Y}(dy), \end{align*} where $\mathbb{P}_{Y} = Y_{*}(\mathbb{P})$ denotes the law of $Y$. This tells us how to define $\mathbb{P}\{X \in A \, \mid \, Y = y\}$ in a way that agrees with the elementary definition.

Answer 2

Using the Lebesgue differentiation theorem, a quite elementary description is that $$ P(x\in A|Y=y)=\lim_{\delta\to0}\frac{P(x\in A,Y\in B_\delta(y))}{P(Y\in B_\delta(y))} $$ for almost every $y$, where $B_\delta(y)=(y-\delta,y+\delta)$.