Evaluating marginal likelihood with Gaussian likelihood and prior

Question

I'm doing Bayesian problem where I need to calculate a posterior function for distribution mean $\theta$.

The problem goes as follows: I have a data set $y_1, y_2, ..., y_n$ with each $y_i \sim N(\theta, \sigma^2 = 40).$ I'm given that $\frac1n\sum_{i=1}^n y_i = 150$ and that the prior distribution for $\theta$ is $p(\theta)=N(\mu_0=180, \tau_0^2=160).$ Derive the posterior distribution for $\theta$, i.e. $p(\theta|y)$.

I have no problem in this task except with evaluating the normalizing constant:

\begin{align}p(y)&=\int_{-\infty}^{\infty}p(y|\theta)p(\theta)\,d\theta\\&=\frac{1}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\int_{-\infty}^{\infty}\exp\left(-\frac{1}{2}\left(\frac{1}{\tau_0^2}(\theta-\mu_0)^2+\frac{1}{\sigma^2} \sum_{i=1}^n(y_i-\theta)^2\right)\right)\,d\theta \end{align}

My question is: how can I evaluate the Gaussian integral above?

Answer 1

\begin{align}p(y)&=\int_{-\infty}^{\infty}p(y|\theta)p(\theta)\,d\theta\\&=\frac{1}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\int_{-\infty}^{\infty}\exp\left(-\frac{1}{2}\left(\frac{1}{\tau_0^2}(\theta-\mu_0)^2+\frac{1}{\sigma^2} \sum_{i=1}^n(y_i-\theta)^2\right)\right)\,d\theta \end{align}

Notice that

$$\frac{1}{\tau_0^2}(\theta-\mu_0)^2+\frac{1}{\sigma^2} \sum_{i=1}^n(y_i-\theta)^2 $$

is a quadratic function with leading coefficient being positive in $\theta$ and hence we can write it in the form of $$\frac{(\theta -B)^2}{A^2} + C$$

Hence \begin{align}p(y)&= \frac{1}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\int_{-\infty}^{\infty}\exp\left(-\frac{(\theta -B)^2}{2A^2} -\frac{C}{2}\right)\,d\theta \\&= \frac{\exp(-C/2)}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\int_{-\infty}^{\infty}\exp\left(-\frac{(\theta -B)^2}{2A^2} \right)\,d\theta \\ &=\frac{\exp(-C/2)}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}} \sqrt{2\pi A^2}\end{align}

Hence if it suffices to solve for $A$ and $C$ to solve for your problem.

Answer 2

By using the answer already given I tried solving it myself also :)

$$\frac{1}{\tau_0^2}(\theta-\mu_0)^2+\frac{1}{\sigma^2} \sum_{i=1}^n(y_i-\theta)^2 =\frac{1}{\tau_0^2}(\theta^2-2\theta\mu_0+\mu_0^2)+\frac{1}{\sigma^2}\left(n\theta^2-2\theta \sum_{i=1}^n y_i+\sum_{i=1}^n y_i^2\right) $$

$$\underbrace{\left(\frac{1}{\tau_0^2}+\frac{n}{\sigma^2}\right)}_{=A}\theta^2-2\underbrace{\left(\frac{\mu_0}{\tau_0^2}+\frac{\sum_{i=1}^n y_i}{\sigma^2}\right)}_{=B}\theta+\underbrace{\left(\frac{\mu_0^2}{\tau_0^2}+\frac{1}{\sigma^2}\sum_{i=1}^n y_i^2\right)}_{=C}=A\theta^2-2B\theta+C,$$

so now I have:

$$\begin{align} p(y) &=\int_{-\infty}^{\infty}p(y|\theta)p(\theta)\,d\theta=\frac{1}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\int_{-\infty}^{\infty}\exp\left(-\frac{1}{2}\left(\frac{1}{\tau_0^2}(\theta-\mu_0)^2+\frac{1}{\sigma^2} \sum_{i=1}^n(y_i-\theta)^2\right)\right)\,d\theta \\ &= \frac{1}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\int_{-\infty}^{\infty}\exp\left(-\frac A2\theta^2+B\theta-\frac C2\right)\,d\theta \\ &= \frac{\exp\left(-\frac C2\right)}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\int_{-\infty}^{\infty}\exp\left(-\frac A2\theta^2+B\theta\right)\,d\theta \\&= \frac{\exp\left(-\frac C2\right)}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}} \int_{-\infty}^{\infty}\exp\left(-\frac A2\left(\theta-\frac BA\right)^2 +\frac{B^2}{2A}\right)\,d\theta \\&= \frac{\exp\left(\frac{B^2}{2A}-\frac C2\right)}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}} \int_{-\infty}^{\infty}\exp\left(-\frac A2\left(\theta-\frac BA\right)^2 \right)\,d\theta\end{align}. $$

Now I set $F=\frac A2\left(\theta-\frac BA\right), dF=\frac A2 d\theta$ so I get:

$$\begin{align} &=\frac{\exp\left(\frac{B^2}{2A}-\frac C2\right)}{\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}} \int_{-\infty}^{\infty}\exp\left(-\frac A2\left(\theta-\frac BA\right)^2 \right)\,d\theta =\frac{2\exp\left(\frac{B^2}{2A}-\frac C2\right)}{A\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}} \int_{-\infty}^{\infty}\exp\left(-F^2 \right)\,dF \\&= \frac{2\exp\left(\frac{B^2}{2A}-\frac C2\right)}{A\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}\sqrt{\pi}\\&= \frac{2\sqrt{\pi}\exp\left(\frac{\left(\frac{\mu_0}{\tau_0^2}+\frac{\sum_{i=1}^n y_i}{\sigma^2}\right)^2}{2\left(\frac{1}{\tau_0^2}+\frac{n}{\sigma^2}\right)}-\frac{\left(\frac{\mu_0^2}{\tau_0^2}+\frac{1}{\sigma^2}\sum_{i=1}^n y_i^2\right)}{2}\right)}{\left(\frac{1}{\tau_0^2}+\frac{n}{\sigma^2}\right)\sqrt{(2\pi\sigma^2)^n2\pi\tau_0^2}}.\end{align}$$