Posterior distribution normal normal bayesian model

133 Views Asked by At

I am trying to understand example from the following lecture: https://people.eecs.berkeley.edu/~jordan/courses/260-spring10/lectures/lecture5.pdf (both $\mu$ and $\sigma$ are unknown - page 5).

They consider: $$ x_i| \mu, \tau \sim \mathcal{N}(\mu, \tau) \text{ iid} $$ $$ \mu|\tau \sim \mathcal{N}(\mu_0, n_0 \tau) $$ $$ \tau \sim Gamma(\alpha, \beta) $$

First, lets look at: $\mu|x, \tau$.

My computations:

$$ f(\mu|x, \tau)\propto \prod \limits_{i=1}^n f(x_i|\mu, \tau) f(\mu|\tau) = \frac{1}{ (2\pi)^{\frac{n+1}{2}} \sqrt{n_0} \tau^{\frac{n+1}{2}} } \exp{ (- \sum \limits_{i=1}^{n}} \frac{n_0 (x_i -\mu)^2}{2\tau n_0} - \frac{(\mu - \mu_0)^2}{2 \tau n_0}) \propto $$ $$ \propto \exp ({\frac{-n_0 \sum x_i^2 + 2n_0\mu \sum x_i - nn_0\mu^2 - \mu^2 + 2\mu\mu_0 - \mu_0^2}{2\tau n_0}} ) $$ $$ \propto\exp {\frac{-\mu^2(1+nn_0) + 2\mu (\mu_0 + n_0 \sum x_i))}{2\tau n_0}} $$ $$ \propto\exp( {- \frac{(\mu - \frac{\mu_0 + n_0 \sum x_i}{1 + n n_0})^2}{2 \frac{\tau n_0}{1+nn_0}} }) $$ Therefore, we would have that:

$$ \mu|x, \tau \sim \mathcal{N}(\frac{\mu_0 + n_0 \sum x_i}{1+nn_0},\frac{\tau n_0}{1+n n_0}) $$ which is sth different than the result obtained in the aforementioned lecture. I think I am doing a mistake at the beginning. Please, help me to find it.

1

There are 1 best solutions below

0
On

The first question I want to ask you is, why would you attempt to perform the calculations from scratch if the notes already tell you how to apply the previous results that were derived? We want $\mu \mid \boldsymbol x, \tau$, which only requires us to rewrite Lemma 6 in terms of precision instead of variance. With $\tau_0 = 1/\sigma_0^2$ and $n\tau = n/\sigma^2$, $$\mu \mid \boldsymbol x, \tau \sim \operatorname{Normal}\left( \frac{\sigma_0^2}{\frac{\sigma^2}{n} + \sigma_0^2} \bar x + \frac{\sigma^2}{\frac{\sigma^2}{n} + \sigma_0^2} \mu_0, \left(\frac{1}{\sigma_0^2} + \frac{n}{\sigma^2} \right)^{\!-1} \right) \\ \sim \operatorname{Normal} \left( \frac{n\tau}{\tau_0 + n\tau} \bar x + \frac{\tau_0}{\tau_0 + n\tau} \mu_0, (\tau_0 + n\tau)^{-1} \right)$$ when $\tau$ is known. This is the result in Equation (20) except I have consistently used the posterior variance rather than the posterior precision as the second parameter of the normal distribution (the notes switch and simply write $\tau_0 + n\tau$ because it is implied that when the posterior hyperparameters are expressed in terms of precision, the normal distribution is parametrized in terms of location and precision).