I am having trouble understanding why a graphical model is normalized (e.g. the joint probability sums to 1). Why it does not need a normalization constant?
In PRML Chapter 8.2, it states that: It is easy to show that the representation on the right- hand side of (8.5) is always correctly normalized provided the individual conditional distributions are normalized.
It is easy to show that eq 8.5 is the joint distribution by definition of the directed graph. However, my problem is that I am using two neural networks to predict probability of two random variables:
P(A,B|C)=P(A|C)P(B|C) # both P are outputs of neural nets with softmax as final layer
A and B are conditional independent given C. However, even though both P(A|C) and P(B|C) are normalized distribution (I use softmax as output layer for both networks), the product of them is clearly not normalized.
The issue is that the only constraint mentioned in this book is all conditional probability are normalized. Given my case, I would love to know what are additional constraints my model needs to be followed.
