Independence of conditional probabilities

48 Views Asked by At

Let $X$ and $Y$ be two random variables, each with range $R\subset\mathbb{N}$ of size $|R|=n\in\mathbb{N}$. Consider the sample space $\Omega$ "generated" by the two variables, i.e. an atomic algebra with atoms $X=k\wedge Y=k'$ for all $k,k'\in R$, closed under (finite) conjunction and negation. Let $p$ be a probability function defined on $\Omega$, and moreover positive everywhere on $\Omega$ (such that all of the conditional probabilities below are well-defined).

Question: I'm wondering how many of the $2n^2$ conditional probabilities $p(X=k|Y=k')$ and $p(Y=k|X=k')$ are at least "partially" independent. That is, how many choices of values (in $]0,1[$) can I make before the rest of the above mentioned conditional probabilities are fully determined?

So far I've only found some obvious upper bounds. For example, for all $k\in R$, $\sum_{k'\in R} p(X=k'|Y=k)=1$ and $\sum_{k'\in R} p(Y=k'|X=k)=1$, which yields $2n$ independent constraints. Hence at most $2n(n-1)$ of the above mentioned conditional probabilities are partially independent. The probability laws also entail other constraints, e.g. $p(X=k|Y=k')p(Y=k')=p(Y=k|X=k')p(X=k')$ for all $k,k'\in R$, and $\sum_{k\in R} p(X=k)=\sum_{k'\in R} p(Y=k') = 1$. But I'm having trouble determining how many independent constraints this gives.

A follow-up question would be what the situation looks like if instead of 2 random variables, I have $m\in\mathbb{N}$ many, each with a range of size $n$.

I would be extremely grateful for any tips/pointers!

1

There are 1 best solutions below

1
On

$$p(X=k|Y=k′) = \frac{p(X=k \ and\ Y=k')}{\sum_{\forall j} p(X=j \ and\ Y=k')}$$

Suppose you want to calculate every $p(X=k|Y=k′)$ and similarly every $p(Y=k'|X=k)$. You would need $p(X=k \ and\ Y=k')$ for all $k$ and $k'$, which consists of $n^2 - 1$ constraints. If you are given a number of $p(X=k|Y=k′)$ or $p(Y=k'|X=k)$ individually, some of them each provides a new piece of "information" while others can be redundant, as you have pointed out. So this may be a direction you can look into.