In a course of Bayesian statistics, I have been given an excercise from Hoff, about the computations of some full conditional for the Probit model.
I have not great confidence with conditional probability computation.
So I would like to ask if the following resoning is correct.
Thanks in advance.
Consider the following Probit model: $$Z_i=\beta x_i+\epsilon_i\qquad\epsilon_1,\dots,\epsilon_n\sim_{iid}\mathcal{N}(0,1)$$ $$Y_i=\mathbb{1}_{(c,+\infty)}(Z_i)$$ where:
- $\beta$,$c$ unknown parameters, for which we assume priors $\beta\sim\mathcal{N}(0,\sigma_\beta^2)$, $c\sim\mathcal{N}(0,\sigma_c^2)$
- the covariates $x_i$ are known and hence tretated as constants
We have to derive $p(\beta\mid z_{1:n}, y_{1:n}, c)$.
Clearly, $c$ and $y_{1:n}$ bring no information on $\beta$. Then:
$$p(\beta\mid z_{1:n},y_{1:n}, c)=p(\beta\mid z_{1:n})\propto p(z_{1:n}\mid\beta)p(\beta)$$ $$\propto\mathtt{exp}(-\frac{1}{2}\{\sum_{i=1}^{n}(z_i-\beta x_i)^2+(\beta/\sigma_\beta)^2\}) \propto \mathtt{exp}[-\frac{1}{2}(\frac{\beta-\tilde{\mu}_\beta}{\tilde{\sigma}_\beta})^2]$$
where: $$\tilde{\sigma}_\beta^2=\left(\sum_{i=1}^n x_i^2+\frac{1}{\sigma^2_\beta}\right)^{-1}\qquad\tilde{\mu}_\beta=\tilde{\sigma}_\beta\left(\sum_{i=1}^n x_iz_i\right)$$ Hence, we conclude $p(\beta\mid z_{1:n}, y_{1:n}, c)\sim\mathcal{N}(\tilde{\mu}_\beta,\tilde{\sigma}_\beta)$
- Question 1 Is this computation correct?
We have to derive $p(c\mid z_{1:n}, y_{1:n}, \beta)$
Clearly, $\beta$ brings no information on $c$.
Moreover, observe that $p(c\mid z_{1:n})=p(c)$, since $z_{1:n}$ alone brings no information on $c$.
Then:
$$ p(c\mid\beta,z_{1:n},y_{1:n})=p(c\mid z_{1:n},y_{1:n})\propto p(y_{1:n}\mid c,z_{1:n})p(c\mid z_{1:n})=p(y_{1:n}\mid c,z_{1:n})p(c)$$
Now, observe that $y_i\mid c,z_i$ is not random any more. In particular it is one if $z_i>c$ and it is $0$ if $z_i\leq c$. Then we may write, observing that $z_{-i}$ brings no information about $y_i$: $$p(y_i\mid c,z_{1:n})=p(y_i\mid c,z_i)=\mathbb{1}_{(z_i>c)}^{y_i}\mathbb{1}_{(z_i\leq c)}^{1-y_i}$$ Then: $$p(y_{1:n}\mid c,z_{1:n})=\prod_{i=1}^{n}\mathbb{1}_{(z_i>c)}^{y_i}\mathbb{1}_{(z_i\leq c)}^{1-y_i}$$ And in turn: $$p(c\mid\beta,z_{1:n},y_{1:n})\propto\left(\prod_{i=1}^{n}\mathbb{1}_{(z_i>c)}^{y_i}\mathbb{1}_{(z_i\leq c)}^{1-y_i}\right)\mathcal{N}(0,\sigma_c^2)$$
- Question 2According to a given hint,parenthesis should reduce to the charachterstic function of an interval, so that we have a truncated gaussian. But I cannot see how it can. can someone help?
We have to derive $p(z_i\mid z_{-i}, y_{1:n}, \beta, c)$. Clearly, $z_{-i}$ brings no information on $z_{-i}$, similarly for $y_{-i}$. $$p(z_i\mid y_{i},\beta,c)\propto p(y_i\mid z_{i},\beta,c)p(c) \propto p(y_i\mid z_{i},c)p(c) \propto \mathbb{1}_{(z_i>c)}^{y_i}\mathbb{1}_{(z_i\leq c)}^{1-y_i}\mathcal{N}(0,\sigma_c^2)$$
- Question 3 Is this computation correct?