Identifying a conditional distribution for Gibbs sampling

80 Views Asked by At

I have $N$ samples with $M$ features with class labels $T\in\{-1, 1\}$ which were generated by drawing each feature $m$ from a normal distribution $N\sim N(0, \sigma_m)$.

Class labels were assigned as follows:

$T=1$ if $Y=\sum_ma_mX_m>0$,

$T=-1$, if $Y\leq0$

with $a_m \in [-1, 1]$, the coefficients were drawn from a uniform distribution.

My goal is to estimate the $a_m$ using Gibbs sampling. To achieve this I need to find the conditional distribution for each $a_m$:

$$P(a_m|Y,\mathbf{x_n}, t_n, \{\sigma_m\},\{a_k\}_{k\neq m}) =\frac{P(Y,\mathbf{x_n}, t_n, \{\sigma_m\},\{a_k\}_{k\neq m}|a_m)*P(a_m)}{P(Y,\mathbf{x_n}, t_n, \{\sigma_m\},\{a_k\}_{k\neq m})}$$

Where $\mathbf{x_n}=(x_1, x_2,...,x_M)$, $t_n$ is the associated class label and $\{a_k\}_{k\neq m}$ is the set of all coefficients that are not $a_m$.

Since all elements are given $P(Y,\mathbf{x_n}, t_n, \{\sigma_m\},\{a_k\}_{k\neq m})$ is just a constant: $$P(a_m|Y,\mathbf{x_n}, t_n, \{\sigma_m\},\{a_k\}_{k\neq m})\propto P(Y,\mathbf{x_n}, t_n, \{\sigma_m\},\{a_k\}_{k\neq m}|a_m)=$$

$$=P(Y|\mathbf{x_n}, t_n, \{\sigma_m\},\{a_k\}_{k\neq m},a_m)*P(\mathbf{x_n}, t_n, \{\sigma_m\},\{a_k\}_{k\neq m}|a_m)$$

$$\propto P(Y|\mathbf{x_n}, t_n, \{\sigma_m\},\{a_k\}_{k\neq m},a_m)$$

Now, assuming $t_n=1$ for this $\mathbf{x_n}$, I know that $Y>0$. I also know that $P(Y|\{a_m\})$ is a Gaussian distribution, but I can't find an expression for $P(Y|\mathbf{x_n}, t_n, \{\sigma_m\},\{a_k\}_{k\neq m},a_m)$ from which I could sample $a_m$. How can I bring all the individual components together or did I make another mistake?