What Information is Needed to Determine the Joint Probability Distribution?

Question

I have just started learning probability theory. I came across this result:

Suppose $X$ and $Y$ are two non-independent random variables defined on same probability space and their individual probability density functions are given, i.e., $f_X(x)$ and $f_Y(y)$. This is not sufficient information to completely determine their joint probability distribution function, i.e., $f_{X,Y}(x,y)$.

Can you give an example that illustrates the above fact? What extra information do we need to calculate the joint PDF?

Answer 1

Define $Q(x,y) \subseteq \mathbb{R}^2$ as a square (including interior points) in $\mathbb{R}^2$ with side length $1$, sides parallel to the $x$- and $y$-axes and center at $(x,y)$.

Case 1: Let $X,Y: \Omega \rightarrow \mathbb{R}$ be real-valued continuous random variables with joint distribution function $ f_{X,Y}(x,y) = 1/2 $ for $ (x,y) \in Q(1/2,1/2) \ \cup Q(3/2,3/2) $ and $f_{X,Y}(x,y) = 0$ otherwise. Then the individual densitiy of $X$ is $f_{X} = 1/2 $ for $x \in [0,2] $ and $f_{X} = 0$ otherwise. The individual densitiy of $Y$ is $f_{Y} = 1/2 $ for $y \in [0,2] $ and $f_{Y} = 0$ otherwise. $X$ and $Y$ are not independent.

Case 2: Let $X,Y: \Omega \rightarrow \mathbb{R}$ be real-valued continuous random variables with joint distribution function $ f_{X,Y}(x,y) = 1/2 $ for $ (x,y) \in Q(1/2,3/2) \ \cup Q(3/2,1/2) $ and $f_{X,Y}(x,y) = 0$ otherwise. Then the individual densitiy of $X$ is $f_{X} = 1/2 $ for $x \in [0,2] $ and $f_{X} = 0$ otherwise. The individual densitiy of $Y$ is $f_{Y} = 1/2 $ for $y \in [0,2] $ and $f_{Y} = 0$ otherwise. $X$ and $Y$ are not independent.

The above two cases show, that the individual densities of two random variables are not enough to determine the joint densitiy, since in both cases you have the same individual densities but different joint density.

Answer 2

Suppose $X$ and $Y$ each take values 0 or 1.

Model 1: Points $(0,0)$ and $(1,1)$ each have probability $1/2.$

Model 2: Points $(0,0), (0,1), (1,0),$ and $(1,1)$ each have probability $1/4.$

Both models have $P(X=0) = P(X=1) = 1/2$ and $P(Y=0) = P(Y=1) = 1/2.$ So the two models have the same marginal distributions. However, in Model 2 $X$ and $Y$ are independent, and in Model 1 they are not.

Answer 3

For any continuous RVs $X$ and $Y$ with marginal CDFs $F_X(x)$ and $F_Y(y),$ it's always a possibility that we have $$ Y =F_{Y}^{-1} (F_X(X)).$$ A good exercise is to check that this is consistent, i.e. if $X$ has CDF $F_X$ then $Y$ as defined above has CDF $F_Y.$ And it's pretty clear that $X$ and $Y$ are not independent (except perhaps in some corner cases).