Consider two different random variables on $\{0,1\}^{\mathbb N}$, i.e. the set of binary sequences. The first random variable $X_1$ has a distribution defined by letting each of its digits be chosen by IID Bernoulli trials $\sim\text{Bern}(0.5)$, so that each one is $0$ with probability $0.5$ and $1$ with probability $0.5$. The second random variable $X_2$ has a distribution defined by letting each digit be chosen by an independent Bernoulli trial $\sim\text{Bern}(0.75)$, so that each one is $0$ with probability $0.75$ and $1$ with probability $0.25$. Note that the asymptotic density of $0$s in $X_1$ will "almost surely" (with probability $1$) be equal to $50\%$, and the asymptotic density of $0$s in $X_2$ will "almost surely" equal $75\%$.
Now, define a third random variable $X$ to be given by the result of $X_1$ with probability $0.5$ and the result of $X_2$ with probability $0.5$. My questions are as follows:
- Given that the result of $X$ is a sequence consisting of $40\%$ zeroes, what is the probability that its value was taken from $X_1$ as opposed to $X_2$?
- Is the above question even well-posed? Or does it violate some part of the technical definition of conditional probability?
The complication here comes from the fact that the probability of $X$ producing a sequence with $40\%$ zeroes is $0$ ("almost certainly will not happen"), meaning that attempting to use the usual conditional probability formula results in a division by zero. Nevertheless, given this "infinitely unlikely" outcome, it seems to me like the "correct answer" should be that $X$ was most likely taken from $X_1$, perhaps even "almost certainly" taken from $X_1$.
By "the result of $X$ is a sequence consisting of 40% zeroes", I assume that you mean that $$\lim_{n \rightarrow \infty} \sum_{i = 1}^n X_{0,i}/n = 0.4,$$ where $X = (X_{0,1}, X_{0,2}, \dots)$.
The event that the sum converges is a tail event, one that is independent of any finite subset of the $X_{0,i}$ values, so by Kolmogorov's zero-one law it has probability $0$ or $1$, and it's clear that it's not $1$, so $$\mathbb{P}\left(\lim_{n \rightarrow \infty} \sum_{i = 1}^n X_{0,i}/n = 0.4\right) = 0.$$
Because the probability of seeing $X$ with 40% zeroes is $0$, the conditional probability is undefined (Wikipedia). Normally in a situation like this you would choose an interval near to $0.4$, e.g. $(0.4 - \epsilon, 0.4 + \epsilon)$, and then condition on $$\lim_{n \rightarrow \infty} \sum_{i = 1}^n X_{0,i}/n \in (0.4 - \epsilon, 0.4 + \epsilon),$$ but this event has probability $0$ as well until we have $\epsilon \geq 0.1$.
If you were seeing that roughly 40% of the $X$ values are zeroes all the way through then you could compute the limit of the conditional probability for the finite case: suppose that we've seen $n$ values and 40% of them were $0$. That is, $\sum_{i=1}^n X_{0,i}/n = 0.4$. Let's denote this event $E_n$. Then by Bayes' Theorem $$\mathbb{P}\left(X = X_1 \mid E_n \right) = \frac{\mathbb{P}\left(E_n \mid X = X_1\right) \mathbb{P}\left(X = X_1\right)}{\mathbb{P}\left(E_n \mid X = X_1\right) \mathbb{P}\left(X = X_1\right) + \mathbb{P}\left(E_n \mid X = X_2\right) \mathbb{P}\left(X = X_2\right)}.$$
Our prior probabilities are $$\mathbb{P}\left(X = X_1\right) = \mathbb{P}\left(X = X_2\right) = 0.5,$$ and $$\mathbb{P}\left(E_n \mid X = X_1\right) = \binom{n}{0.4n} 0.5^{0.4n} 0.5^{0.6n},$$ and $$\mathbb{P}\left(E_n \mid X = X_2\right) = \binom{n}{0.4n} 0.75^{0.4n} 0.25^{0.6n},$$ and putting this together and cancelling out the common terms gives \begin{align*} \mathbb{P}\left(X = X_1 \mid E_n\right) &= \frac{0.5^{0.4n} 0.5^{0.6n}}{0.5^{0.4n} 0.5^{0.6n} + 0.75^{0.4n} 0.25^{0.6n}}\\ &= \frac{1}{1 + (0.75/0.5)^{0.4n} (0.25/0.5)^{0.6n}}\\ &= \frac{1}{1 + 1.5^{0.4n} 0.5^{0.6n}}\\ &= \frac{1}{1 + 0.75^{0.4n} 0.5^{0.2n}} \end{align*} Therefore $$\lim_{n \rightarrow \infty} \mathbb{P}\left(X = X_1 \mid E_n\right) = 1.$$ To be clear, this isn't the exact conditional probability that you were looking for but I think it's as close as you can get.