Calculate the posterior

106 Views Asked by At

I have the following old exam task and I need some help to understand how to solve it. I mean I know how to solve something like that in theory, but their has to be a trick, especially for the integral. I can not believe that they let us calculate something like that "pure" in the exam.

Following task: Compute the (normalized) posterior, given the data set $\{x^i\}^n_{i=1}$, the likelihood and the prior. $$p(x|\mu)= \frac{1}{\sqrt{2 \pi \delta^2}}\exp(-\frac{(x-\mu)^2}{2\delta^2}) $$ $$ p(\mu) = \frac{1}{\sqrt{2 \pi a^2}}\exp(-\frac{\mu^2}{2a^2})$$

1

There are 1 best solutions below

0
On BEST ANSWER

Working with the kernels makes the computations much easier. Let $\boldsymbol x = (x_1, \ldots, x_n)$ be the sample. Then $$p(\boldsymbol x \mid \mu) \propto \exp \left( - \frac{1}{2\delta^2} \sum_{i=1}^n (x_i - \mu)^2 \right). \tag{1}$$ We also have $$p(\mu) \propto \exp \left(- \frac{\mu^2}{2a^2} \right). \tag{2}$$

So the posterior is proportional to $$p(\mu \mid \boldsymbol x) \propto p(\boldsymbol x \mid \mu)p(\mu) = \exp \left( - \frac{1}{2\delta^2} \sum_{i=1}^n (x_i - \mu)^2 - \frac{\mu^2}{2a^2}\right). \tag{3}$$

Let's decompose the sum of squares in $(3)$. Consider the expression

$$\begin{align} \sum_{i=1}^n (\mu - x_i)^2 + \frac{\delta^2 \mu^2}{a^2} &= \sum_{i=1}^n (\mu^2 - 2\mu x_i + x_i^2) + \frac{\delta^2 \mu^2}{a^2} \\ &= \left(n + \frac{\delta^2}{a^2}\right) \mu^2 - 2n \bar x \mu + \sum_{i=1}^n x_i^2 \\ &= A\mu^2 - B\mu + C, \tag{4} \end{align}$$ where $$A = n + \frac{\delta^2}{a^2}, \quad B = 2n\bar x, \quad C = \sum_{i=1}^n x_i^2. \tag{5}$$ We seek to complete the square of $(4)$: $$A\mu^2 - B\mu + C = A\left(\mu^2 - \frac{B}{A}\mu + \frac{B^2}{4A^2}\right) + C - \frac{B^2}{4A} = A\left(\mu - \frac{B}{2A}\right)^2 + K, \tag{6}$$ where $K = C - B^2/(4A)$ is a constant with respect to $\mu$. Therefore, $$\begin{align} p(\mu \mid \boldsymbol x) &\propto \exp \left( - \frac{A}{2\delta^2} \left(\mu - \tfrac{B}{2A}\right)^2 - \frac{K}{2\delta^2}\right) \\ &= \exp \left( - \frac{(\mu - B/(2A))^2}{2 \delta^2/A} \right) \exp \left(-\frac{K}{2\delta^2}\right) \\ &\propto \exp \left( - \frac{(\mu - B/(2A))^2}{2\delta^2/A}\right). \tag{7} \end{align}$$ As this is the kernel of a normal distribution, the posterior is normally distributed with mean $$\frac{B}{2A} = \frac{n \bar x}{n + (\delta/a)^2}, \tag{8}$$ and variance $$\frac{\delta^2}{A} = \frac{\delta^2}{n + (\delta/a)^2} = \frac{1}{\frac{n}{\delta^2} + \frac{1}{a^2}}. \tag{9}$$