How to find a probability given other probability functions?

Question

I have the following problem: consider 4 discrete RVs $a,b,c,d$ with $a, b$ independent. The following pmfs are known: $p(a)$, $p(b)$, $p(c|a,b,d)$, $p(d|a,b,c)$ for every value of $a,b,c,d$. Find $p(a|b,c,d)$.

---

My first instinct is to go on a wild goose chase with Bayes rule. For example, I can find $p(a, c | b, d) = p(a)p(c|a,b,d)$ etc. But I just cannot seem to find a way to find $p(a|b,c,d)$. How to find $p(a|b,c,d)$? Is it even computable?

In general, given a list of prior probability functions, how do I know which probability functions are computable?

Answer 1

There's sufficient information in the specified quantities and independence condition to derive the joint probability mass function of all four random variables. Once you have that available you can, of course, derive any of its marginal or conditional probability mass functions. To avoid possible ambiguities of notation or confusion between random variables and their realised values, I'll denote the former by $\ A, B, C\ $ and $\ D\ $ and the latter by corresponding lower case letters. We then have the following quantities given: \begin{align} P(A=a,B=b)&= p(a)p(b)\\ \frac{P(A=a,B=b,C=c,D=d\,)}{P(A=a,B=b,D=d\,)}&=p(c|a,b,d\,)\\ \frac{P(A=a,B=b,C=c,D=d\,)}{P(A=a,B=b,C=c)}&=p(d|a,b,c) \end{align} Taking the ratios of both sides of these last two equations gives us $$ \frac{P(A=a,B=b,C=c)}{P(A=a,B=b,D=d\,)}=\frac{p(c|a,b,d\,)}{p(d|a,b,c)}\ , $$ and then summing this equation over all values of $\ c\ $ gives \begin{align} \sum_c\frac{p(c|a,b,d\,)}{p(d|a,b,c)}&=\sum_c\frac{P(A=a,B=b,C=c)}{P(A=a,B=b,D=d\,)}\\ &=\frac{p(a)p(b)}{P(A=a,B=b,D=d\,)}\ . \end{align} or $$ P(A=a,B=b,D=d\,)=\frac{p(a)p(b)}{\sum_\limits{c}\frac{p(c|a,b,d\,)}{p(d|a,b,c)}}\ . $$ Now substituting this expression for $\ P(A=a,B=b,D=d\,)\ $ into the second equation above gives \begin{align} P(A=a,B=b,C=c,D=d\,)&=P(A=a,B=b,D=d\,)p(c|a,b,d\,)\\ &=\frac{p(a)p(b)p(c|a,b,d)}{\left(\sum_\limits{c}\frac{p(c|a,b,d)}{p(d|a,b,c)}\right)}\ . \end{align} The quantity you're asked to find can now be expressed as \begin{align} P(A=a\,|B=b,C=c,D=d\,)&=\frac{P(A=a,B=b,C=c,D=d\,)}{P(B=b,C=c,D=d\,)}\\ &=\frac{P(A=a,B=b,C=c,D=d\,)}{\sum_\limits{a}P(A=a,B=b,C=c,D=d\,)}\\ &=\frac{p(a)p(b)p(c|a,b,d)}{\left(\sum_\limits{c}\frac{p(c|a,b,d)}{p(d|a,b,c)}\right)\left(\sum_\limits{a}\frac{p(a)p(b)p(c|a,b,d)}{\sum_\limits{c}\frac{p(c|a,b,d)}{p(d|a,b,c)}}\ \ \right)}\ . \end{align}