Conditional probability where the conditioning variable is continuous

Question

Consider

• a random variable $Y$ with finite support $\mathcal{Y}$

• a random variable $X$ with cdf $G$ absolutely continuous with probability density function $g$ (i.e., $X$ is a continuous random variable with support $\mathcal{X}$)

• all random variables are defined on the probability space $(\Omega, \mathcal{F}, \mathbb{P})$

For some $y\in \mathcal{Y}$, let $$ \mathbb{P}(Y=y| X)\equiv h_y(X) $$ where $h_y: \mathcal{X}\rightarrow [0,1]$

What is the function $$ (y,x)\in\mathcal{Y} \times \mathcal{X} \mapsto h_y(x)* g(x)\in \mathbb{R} $$ ? ($*$ denotes scalar multiplication)

Is it the joint probability density function of $(Y,X)$?

Answer 1

There is no (traditional, i.e., non-impulsive) joint density for $(X,Y)$ because $Y$ is a discrete random variable. Nevertheless, the function $d(x,y) = P[Y=y|X=x]f_X(x)$ can be viewed "operationally" as having the same desirable features of a density, with the understanding that we "sum" over the $y$ variable, not integrate.

• For example, we "sum out $y$" to get the marginal density for $X$: $$\sum_{y \in \mathcal{Y}} P[Y=y|X=x] f_X(x) = f_X(x)$$ This is distinct from "integrating out" the $y$ variable if there were a tranditional density $f_{XY}(x,y)$, i.e., the standard formula $f_X(x) = \int_{y=-\infty}^{\infty} f_{XY}(x,y)dy$.

• Also note that we can "switch the conditioning" as desired: $$ P[Y=y|X=x]f_X(x) = f_{X|Y}(x|y)P[Y=y]$$

• Finally, for any measurable set $A \subseteq \mathbb{R}^2$, if we let $1_{\{(x,y) \in A\}}$ be an indicator function that is 1 if $(x,y) \in A$, and zero else, then $$\boxed{P[(X,Y) \in A] = \sum_{y\in \mathcal{Y}} \int_{x=-\infty}^{\infty} P[Y=y|X=x]f_X(x) 1_{\{(x,y) \in A\}} dx}$$ Indeed \begin{align} P[(X,Y)\in A] &= \int_{x=-\infty}^{\infty} P[(X,Y)\in A| X=x] f_X(x)dx\\ &=\int_{x=-\infty}^{\infty} P[(x,Y) \in A|X=x]f_X(x)dx\\ &\int_{x=-\infty}^{\infty} \left(\sum_{y \in \mathcal{Y}} P[Y=y|X=x]1_{\{(x,y)\in A\}}\right) f_X(x)dx \end{align} and we can formally switch sums/integrals by Fubini-Tonelli for this non-negative function.

Answer 2

This relates to a concept I have though about a lot; what follows includes some opinions.

Let $f(x,y)=h_y(x)\cdot g(x)$. You would not call $f$ a joint density for $(X,Y)$. That term is reserved for the situation where $(X,Y)$ is absolutely continuous with respect to Lebesgue measure $\lambda$ on the plane. That is, we cannot say that $$ \mathbb P((X,Y)\in A)=\int_A f(x,y)\,d\lambda \tag{1} $$ However, $f$ certainly looks like a density in the following sense. Let $H^1$ denote $1$-dimensional Hausdorff measure on $\mathbb R^2$. Then for any measurable $A$, $$ \mathbb P((X,Y)\in A)=\int_A f(x,y)\,dH^1\tag{2} $$ In the situation (1), the support of $(X,Y)$ is a two-dimensional set, wheras in your situation (2), the support is a one dimensional set, a finite union of several lines $\{y\}\times \mathcal X$ for $y\in \mathcal Y$. In other words, if you generalize the notion of "density" to allow for other supports with other dimensions by integrating with respect to Hausdorff measure $H^d$, then you can call your function a (generalized) density. I think that we should adopt this convention, but as far as I know no one does.

Another example: if $X$ has the Cantor distribution, then the function $f(x)={\bf 1}(x\in C)$, where $C$ is the Cantor set, can be viewed as a $(\log_3 2)$-dimensional density function$^*$ for $X$. We then have $$ P(X\in A)=\int_A f(x)\,dH^{\log_3 2} $$ This illustrates that $X$ is uniformly distributed over the Cantor set, since the density function is constant.

$^*$ Perhaps you need a normalizing constant to make to make this integrate to $1$.