Conjugate Prior for $\sigma^2$ is Inverted Gamma; Find the Posterior Distribution of $\sigma^2$

1.3k Views Asked by At

If $S^2$ is the sample variance based on a sample of size $n$ from a normal population, we know that $\frac{(n-1)S^2}{\sigma^2}$ has a $X^2(n-1)$ distribution. The conjugate prior for $\sigma^2$ is the inverted gamma pdf, $IG(\alpha, \beta)$, given by:

$$\pi(\sigma^2)=\frac{e^{-(\frac{1}{\beta\sigma^2})}}{\Gamma(\alpha)\beta^\alpha(\sigma^2)^{\alpha+1}}$$

where $\alpha$ and $\beta$ are positive constants.

Show that the posterior distribution of $\sigma^2$ is:

$$IG\left(\alpha+\frac{n-1}{2},\left[ \frac{(n-1)S^2}{2} + \frac{1}{\beta} \right]^{-1} \right)$$

Here's what I've got so far:

So I need show that $f(\sigma^2|?)\propto\pi(\sigma^2)f(?|\sigma^2)$

The only problem is, I'm not sure what $?$ is in the context of this problem. Without even knowing that, I can't possibly proceed. Any advice?

1

There are 1 best solutions below

2
On BEST ANSWER

The model tends to assume that $\mu$ is constant, and we model $\sigma^2$ as an inverse Gaussian, so that we have:

$$ X | \sigma^2 \sim N(\mu, \sigma^2), \quad \sigma^2 \sim IG(\alpha, \beta) $$

The posterior of $\sigma^2$ is given as:

\begin{align*} p(\sigma^2 | X) & = P(X|\sigma^2) p(\sigma^2)\\ &= \frac{1}{\sqrt{2 \pi} \sqrt{\sigma^2}} e^{\frac{1}{2} \frac{(x-\mu)^2}{\sigma^2}} \frac{1}{\Gamma(\alpha) \beta^{\alpha} (\sigma^2)^{\alpha+1}} e^{\frac{-1}{\beta \sigma^2}} \\ &\propto \frac{1}{\sqrt{\sigma^2}}\frac{1}{(\sigma^2)^{\alpha + 1}} \exp \left ( -\frac{1}{2} \left ( \frac{(x-\mu)^2}{\sigma^2} + \frac{2}{\beta \sigma^2}\right ) \right ) \\ &=\frac{1}{(\sigma^2)^{\alpha + 1+1/2}} \exp \left ( \frac{\beta (x-\mu)^2 + 2}{2 \beta } \frac{1}{\sigma^2}\right )\\ &=\frac{1}{(\sigma^2)^{\alpha' + 1}} \exp \left ( \frac{1}{\beta' } \frac{1}{\sigma^2}\right )\\ \end{align*}

where $\alpha' := \alpha + 1/2$ and $\beta' = \frac{2\beta}{\beta (x-\mu)^2 + 2}$

Notice that the final expression is the kernel of an inverse gamma with parameters $\alpha', \beta'$.

In the case of $n$ samples as you have in your question, we need to change

$$ P(X | \sigma^2) = \prod_{i=1}^n \mathcal{N}(X_i | \mu, \sigma^2) $$

The solution should come out in a similar fashion