I wanted to verify my approach to marginalizing out variables in a Bayes' net or Probabilistic Graphical model. Consider the following model:
$$\require{enclose}\begin{array}{c} \enclose{circle}{~\text{Burglary}~}&&&&\enclose{circle}{~\text{Fire}~}\\ \small\begin{array}{c}\mathsf P({+}B)\\\hline0.001\end{array}&\huge\searrow&&\huge\swarrow&\small\begin{array}{c}\mathsf P({+}F)\\\hline0.002\end{array}\\&&\enclose{circle}{~\text{Alarm}~}\\ &\huge\swarrow&&\huge\searrow&\\ \enclose{circle}{~\text{Police Call}~}&&&&\enclose{circle}{\text{Mary Calls}}&\\ \small\begin{array}{c}A&\mathsf P({+}P\mid A)\\\hline{+}A&0.99\\{-}A&0.005\end{array}&&\small\begin{array}{c}B&F&\mathsf P({+}A\mid B,F)\\\hline {+}B&{+}F&0.95\\{+}B&{-}F&0.94\\{-}B&{+}F&0.29\\{-}B&{-}F&0.0001\end{array}&&\small\begin{array}{c}A&\mathsf P({+}M\mid A)\\\hline{+}A&0.70\\{-}A&0.10\end{array}\end{array}$$
And suppose I want to calculate $\mathbb{P}[+A]$. I was wondering if one could do the following:
$P(+A) = \sum_{P, M, B, F}P(P, M, +A, B, F) =^{by PGM} \sum_{P, M , B, F}P(P|A+)P(M|+A)P(+A|B, F)P(B, F)$
So far so good I think but here's the part I'm less sure of:
$ = \sum_{ B, F}P(+A|B, F)P(B)P(F)\sum_P P(P|+A)\sum_MP(M|+A)$
Now I think we just use that $\sum_P P(P|+A) = \sum_M P(M|+A) = 1$ to get:
$= \sum_{B, F}P(+A|B, F)P(B)P(F)$ and at this point we substitute the values above. Is that the right idea?
Then the next question is how to calculate $P(+B, +F|+A)$. This one I'm a bit more unsure about. I was thinking it might be an application of Bayes' rule:
$P(+B, +F|+A) = \frac{P(+B, +F, +A)}{P(+A)}$ the denominator we already know and the numerator is $P(+B, +F, +A) = P(+A|+B, +F)P(+B)P(+F)$? Is this correct?
And finally, I wanted $P(+B|+A)$ and was thinking of Bayes again but this time:
$P(+B|+A) = \frac{\sum_FP(+B, F, +A)}{P(+A)}$ and the numerator splits up as $\sum_{F}P(+A|+B, F)P(+B)P(F)$
I was wondering if someone could confirm this for me.
Yes, the DAG indicates: $$\mathsf P(^{+}A) = \sum_{B,F}\mathsf P(^{+}A\mid B,F)\,\mathsf P(B)\,\mathsf P(F)$$
Which is either read directly from the diagram, or by distributing the whole factorisation to apply the Law to Total Probability:$$\begin{align}\mathsf P(^{+}A) &=\sum_{B,F,P,M}\mathsf P(P\mid{^{+}A})\,\mathsf P(M\mid {^{+}A})\, \mathsf P({^{+}A}\mid B,F)\,\mathsf P(B)\,\mathsf P(F)\\[1ex] &= \sum_{B,F}\mathsf P(^{+}A\mid B,F)\,\mathsf P(B)\,\mathsf P(F)\sum_{P}\mathsf P(P\mid{^{+}A})\sum_M\mathsf P(M\mid{^{+}A})\\[1ex] &= \sum_{B,F}\mathsf P(^{+}A\mid B,F)\,\mathsf P(B)\,\mathsf P(F)\end{align}$$
... as you did.
Then you do have: $$\mathsf P({^{+}B},{^{+}F}\mid{^{+}A})=\dfrac{\mathsf P({^{+}A}\mid{^{+}B},{^{+}F})\,\mathsf P(^{+}B)\,\mathsf P(^{+}F)}{\sum_{B,F}\mathsf P(^{+}A\mid B,F)\,\mathsf P(B)\,\mathsf P(F)}$$
And so too$$\mathsf P({^{+}B}\mid{^{+}A})=\dfrac{\mathsf P(^{+}B)\sum_F\mathsf P({^{+}A}\mid{^{+}B},F)\,\mathsf P(F)}{\sum_{B,F}\mathsf P(^{+}A\mid B,F)\,\mathsf P(B)\,\mathsf P(F)}$$