Product of Gaussian densities, why don't the formulas apply to this case?

90 Views Asked by At

I was reading the paper 'A Bayesian tutorial for data assimilation' from Wikle and Berliner and there is this paragraph.

enter image description here

I don't understand how they get to the distribution from X|y. I expected the distribution to be $X|y \sim N\Big( (\frac{1}{\sigma^2} +\frac{1}{\tau ^2})^{-1}(\frac{x}{\sigma^2} +\frac{\mu}{\tau^2}) , (\frac{1}{\sigma^2} +\frac{1}{\tau^2})^2\Big)$,

because that is what you get when you fill in the formula for the product of gaussian densities.

$\mathcal{N}_x(\mu_1 , \Sigma_1)\mathcal{N}_x(\mu_2 , \Sigma_2) = \mathcal{N}_x(\mu_3 , \Sigma_3)$

With

$ \mu_3 = (\Sigma_1^{-1} + \Sigma_2^{-1})^{-1} (\Sigma_1^{-1}\mu_1 +\Sigma_2^{-1}\mu_2)$

$\Sigma_3 = (\Sigma_1^{-1} +\Sigma_2^{-1})^{-1}$

I used $\Sigma_1 = \sigma^2$ , $\Sigma_2 = \tau^2$ , $\mu_1 = x$ and $\mu_2= \mu$. I got these values from $X \sim N(\mu, \tau^2)$ and $Y_i |X = x \sim N(x ,\sigma)^2$.

Could someone explain why these formulas don't work for this case?

( I also tried completing the square, as they say in the paper, but then I also don't get their results)

1

There are 1 best solutions below

2
On BEST ANSWER

Regarding the complete the square part. You have: $$ x^2(n/\sigma^2+1/\tau^2) - 2\left(\sum y_i/\sigma^2+\mu/\tau^2\right)x = ax^2-2bx $$ by defining: $$ a = (n/\sigma^2+1/\tau^2),\ \ \ \ b=\left(\sum y_i/\sigma^2+\mu/\tau^2\right) $$

Now, recall the completing the square trick says that: $$ a(x-b/a)^2-b^2/a = ax^2-2bx+b^2/a-b^2/2 = ax^2-2bx $$ Hence, $$ \begin{aligned} &\exp\left(-0.5\left[x^2(n/\sigma^2+1/\tau^2) - 2\left(\sum y_i/\sigma^2+\mu/\tau^2\right)x\right]\right)\\ & = \exp\left(-0.5\left[ax^2+bx\right]\right)\\ &= \exp\left(-0.5\left[a(x-b/a)^2-b^2/a\right]\right)\\ &=\exp\left(-0.5\left[a(x-b/a)^2\right]\right)\exp\left(0.5b^2/a\right)\\ &\propto \exp\left(-0.5\left[a(x-b/a)^2\right]\right) \\ &= exp\left(-\frac{(x-b/a)^2}{2a^{-1}}\right)\\ &= exp\left(-\frac{(x-\mu')^2}{2{\sigma'}^2}\right) \end{aligned} $$ with $$ \begin{aligned} \mu' &= b/a = (n/\sigma^2+1/\tau^2)^{-1}\left(\sum y_i/\sigma^2+\mu/\tau^2\right)\\ {\sigma'}^2 &= a^{-1} = (n/\sigma^2+1/\tau^2)^{-1} \end{aligned} $$

I don't know where your procedure might have gone wrong, since I don't understand which two Gaussian densities you were multiplying, but your result is clearly wrong: a factor of $n$ should appear somewhere due to the presence of the original $n$ Gaussian densities coming from the $n$ samples. Moreover, $x$ cannot appear as a parameter of the Gaussian density as in your result.

I hope this helps!