Mixture of densities vs mixture of random variables

137 Views Asked by At

Let $f(x),g(x),h(x)$ be probability density functions. Assume that $f$ is a (nontrivial) mixture of $g$ and $h$, that is, there is some $p\in (0,1)$, such that $$f(x)=pg(x)+(1-p)h(x)\hspace{20mm} (*)$$ holds. Let $X$ be a random variable with density $f(x)$.

Question: Does ($*$) imply that there are always two random variables $Y,Z$ with densities $g(x),h(x)$, and a 0-1 valued random variable $\xi$, independent of $Y,Z$, with $\Pr(\xi=1)=p$, $\Pr(\xi=0)=1-p$, such that $$\Pr{\Large (}X=\xi Y+(1-\xi)Z{\Large )}\;=\;1\hspace{20mm} (**)$$ holds? In other words, is it true that if the density of $X$ is a mixture of two other densities, then $X$ can always be realized as mixture of two other random variables with the respective densities, in the sense of ($**$)?

Note: If we just take any $Y,Z,\xi$ with the above properties, then the random variable $$\widetilde X=\xi Y+(1-\xi)Z$$ clearly has distribution $f(x)=pg(x)+(1-p)h(x)$, as $X$. However, nothing guarantees that $\Pr(\widetilde X=X)=1$. How to choose $Y,Z,\xi\;$ if we actually want to realize $X$, not just any other random variable with the same distribution?

1

There are 1 best solutions below

3
On

You are right that $\widetilde{X}$ and $X$ have the same distribution $f$ but they are not necessary equal. That's what i.i.d. variables are all about.

To your original question: I suppose it depends on your probability space, what qualifies as events, what qualifies as r.v., etc. I am not a theorist so I cannot answer that rigorously. As a practical matter, a density of form $f$ would usually arise because the author has some underlying $Y, Z, \xi$ in mind. But as a theoretical matter, whether you can retroactively construct $Y,Z,\xi$... On the same probability space, the answer IMHO is No. On some expanded space (in a sense)... that I don't know.

Here's a discrete example, so we're dealing with pmf as opposed to pdf... hope such a change is OK with you? Consider a fair coin flip, so $P(X=0) = P(X=1) = 1/2$, i.e. its pmf is the vector $[\frac12, \frac12]$. This is a mixture:

$$[\frac12, \frac12] = \frac12 \times [\frac13, \frac23] + \frac12 \times [\frac23, \frac13]$$

However, the original probability space has just two sample points $\Omega = \{Heads, Tails\}$, and using those alone you cannot define any r.v. (in the sense of a function $\Omega \to \mathbb{R}$) with pmf $[\frac13, \frac23]$, so in that sense the r.v. $Y$ where $P(Y=0) = 1/3, P(Y=1)=2/3$ doesn't exist, and $X$ cannot be "realized" as a mix of $Y$ and $Z$ if we restrict to the same space. Now of course you can invent a new probability space, with two biased coins, to implement all of $Y, Z, \xi$.