Derive the posterior full conditional distributions of parameters of linear mixed model

243 Views Asked by At

Consider a linear mixed model:

$y_{ij}=\beta_0+\beta_1x_{ij}+\alpha_i+\epsilon_{ij}, j = 1, ...,n_i, i=1, ..., K$, where $\alpha_i$ is the random effect and $\epsilon_{ij}\sim N(0, \sigma^2)$ is the random error. The prior distributions are: $p(\beta_0)=p(\beta_1)\propto 1$, $\alpha_i \sim N(0, \tau^2)$, $\sigma^2 \sim \text{Inverse-Gamma}(a, b)$, and $\tau^2 \sim \text{Inverse-Gamma}(c, d)$.

I am trying to derive the posterior full conditional distributions for $\beta_0$, $\beta_1$ $\alpha_i$, $\sigma^2$ and $\tau^2$.

For $i=1,...,K$, denote $y_i=\{y_{i1}, ..., y_{in_i}\}$, $x_i=\{x_{i1}, ..., x_{in_i}\}$.

Then, $p(y_i|\beta_0,\beta_1,x_i,\alpha_i,\sigma^2) \propto \Pi_{j=1}^{n_i}p(y_ij|\beta_0, \beta_1, x_ij, \alpha_i) \propto (\sigma^2)^{-n_i/2}\text{exp}(-\frac{1}{2\sigma^2}\cdot\sum_{j=1}^{n_i}(y_{ij}-\beta_0-\beta_1 x_{ij}-\alpha_i)^2)$.

And, there are:

$p(\beta_0)\propto1$, $p(\beta_1)\propto1$,

$p(\alpha_i, \tau^2|c, d)\propto p(\alpha_i|\tau^2)p(\tau^2|c,d)\propto(\tau^2)^{-(c+1/2)-1}\cdot\text{exp}(-\frac{d+\alpha_i^2/2}{\tau^2})$.

$p(\sigma^2|a, b) \propto (\sigma^2)^{-a-1}\text{exp}(-b/\sigma^2)$,

Then, there is

$p(\beta_0, \beta_1, \alpha_i, \sigma^2, \tau^2|x_i,y_i, a, b, c,d) \propto p(y_i|\beta_0, \beta_1, x_i, \alpha_i, \sigma^2)\cdot p(\beta_0)\cdot p(\beta_1) \cdot p(\alpha_i, \tau^2|c,d)\cdot p(\sigma^2|a,b) \propto$

$(\sigma^2)^{-n_i/2}\text{exp}(-\frac{1}{2\sigma^2}\cdot \sum_{j=1}^{n_i}(y_{ij}-\beta_0-\beta_1 x_{ij}-\alpha_i)^2) \cdot (\tau^2)^{-(c+1/2)-1}\cdot\text{exp}(-\frac{d+\alpha_i^2/2}{\tau^2})\cdot (\sigma^2)^{-a-1}\text{exp}(-b/\sigma^2)$.

I know that we can get posterior full conditional distributions by only considering the corresponding terms. And I derive

$p(\beta_0|\beta_1,\alpha_i, \sigma^2, x_i, y_i) \propto \text{exp}(-\frac{1}{2\sigma^2}\cdot \sum_{j=1}^{n_i}(y_{ij}-\beta_0-\beta_1 x_{ij}-\alpha_i)^2)$, and we can symplify it to

$p(\beta_0|\beta_1,\alpha_i, \sigma^2, x_i, y_i) \propto \text{exp}(-\frac{n_i(\beta_0-\bar{\lambda})}{2\sigma^2})$, where $\bar{\lambda}=\frac{1}{n_i}\sum_{j=1}^{n_i}(y_{ij}-\beta_1 x_{ij}-\alpha_i)$.

I am not sure if my process is correct, and I am confused by multivariate distributions. And I have difficulties to derive other full conditional distributions.