I would like to check if I correctly derived my the expression for $p(\mathbf{s}|\mathbf{d})$. Here's the question:
Consider a model of diseases and symptoms. $s_i\in\{0,1\}$ is a binary random variable indicating whether the patient is showing the $i$-th symptom and $d_j\in \{0,1\}$ is a binary random variable indicating whether the patient has $j$-th disease. A model for this is given by $$p(\mathbf{s,d})=\frac{1}{Z}\exp(\mathbf{s}^T\mathbf{W}\mathbf{d}+\mathbf{a}^T\mathbf{s}+\mathbf{b}^T\mathbf{d})$$
where $Z$ is the normalisation constant and $\mathbf{W},\mathbf{a},\mathbf{b}$ are the parameters of the model.
I would like to draw the Markov Network for this model.
Question 1: Would this Markov Network be correct? Since it's a Markov Network, I don't have arrows, otherwise if it was a belief network, here's how I would draw it:
Question 2 Would this belief network be correct?
So now, I have the question:
Derive the expression for $p(\mathbf{s}|\mathbf{d})$. Is this distribution factorised?
Here's my solution:
$$p(\mathbf{s}|\mathbf{d}) = \frac{p(\mathbf{s},\mathbf{d})}{p(\mathbf{d})} = \frac{\exp\left(\mathbf{s}^T\mathbf{W}\mathbf{d}+\mathbf{a}^T\mathbf{s}+\mathbf{b}^T\mathbf{d}\right)}{\sum_{\mathbf{s}}\exp\left(\mathbf{s}^T\mathbf{W}\mathbf{d}+\mathbf{a}^T\mathbf{s}+\mathbf{b}^T\mathbf{d}\right)} = \frac{\exp\left(\mathbf{s}^T\mathbf{W}\mathbf{d}+\mathbf{a}^T\mathbf{s}\right)}{\sum_{\mathbf{s}}\exp\left(\mathbf{s}^T\mathbf{W}\mathbf{d}+\mathbf{a}^T\mathbf{s}\right)} \\ \propto \exp\left(\mathbf{s}^T\mathbf{W}\mathbf{d}+\mathbf{a}^T\mathbf{s}\right) = \exp(\mathbf{s}^T\mathbf{W}d+\mathbf{s}^T\mathbf{a}) = \exp(\mathbf{s}^T(\mathbf{W}\mathbf{d}+\mathbf{a})) = \\ \exp\left(\sum_i^n\mathcal{1}_{\{s_i=1\}} \Big(\sum_j^m W_{ij}d_j+a_i \Big) \right) \\ \prod^n_i \exp(\sum_j^mW_{ij}d_j+a_i) = \prod_i^np(s_i|\mathbf{d}) $$
The distribution is factorised and $p(s_i|\mathbf{d}) = \sigma(a_i+\sum_jW_{ij}d_j)$ where $\sigma(x) = e^x/(1+e^x)$
Is my derivation correct?
No, $p(s|d)$ is proportional to $$\exp\Big( \sum_{i=1}^n s_i \big(\sum_{j=1}^m W_{ij}d_j+a_i\big) \Big) = \prod_{i=1}^n \exp\Big(s_i \big( \sum_{j=1}^m W_{ij}d_j+a_i\big) \Big).$$The distribution is still factorized and $p(s_i|d)$ is proportional to $$\exp\Big( s_i \big( \sum_{j=1}^m W_{ij}d_j+a_i\big) \Big),$$ so it equals $$\frac{\exp\Big( s_i \big( \sum_{j=1}^m W_{ij}d_j+a_i\big) \Big)}{1+\exp\Big( \sum_{j=1}^m W_{ij}d_j+a_i \Big)}.$$