Find posterior distribution and find marginal distribution

107 Views Asked by At

We have z = H s + $\epsilon$ with $\epsilon \sim N(0, R)$ and $s \sim N(\mu, Q)$ where R and Q are symmetric and positive definite. I'm trying to derive the posterior distribution. I have p(s) and p(z|s). However I'm unsure of how to get p(z), and the marginal distribution of y so I can get the posterior distribution p(s|z) using bayes formula. Also R nxn. Q is nxn. H is nxn. s is nx1 and $\epsilon$ is nx1.

p(s) = $\frac{e^{-1/2(s-\mu)^TQ^{-1}(s-\mu)}}{(2*\pi)^{n/2}|Q|^{1/2}} $

p(z|s) = $\frac{e^{-1/2(z-Hs)^TR^{-1}(z-Hs)}}{(2*\pi)^{n/2}|R|^{1/2}} $

Any help would be appreciated.

1

There are 1 best solutions below

7
On

When you find a posterior, it is true that $p(s|z)=\frac{p(z|s)p(s)}{p(z)}$ by Bayes rule. To do this, you would need to calculate the integral $p(z)=\int_{\Omega}p(z|s)p(s)ds$, however this turns out to be a constant (does not depend on s because it has been marginalized out) that will be part of the normalizing factor; consequently, it is more common to use the fact that $p(s|z)\propto p(z|s)p(s)$ to find the posterior density. Here, we can write

$$\begin{split}p(s|z)&\propto p(z|s)p(s)\\ &\propto\exp-\frac 1 2 (s^T-\mu^T)Q^{-1}(s-\mu)\exp-\frac 1 2 (z^T-s^TH^T)R^{-1}(z-Hs)\end{split}$$

Focus on the part inside the exponentiation

$$\begin{split}s^TQ^{-1}s-2\mu^TQ^{-1}s+s^TH^TR^{-1}Hs-2z^TR^THs=\\ s^T\underbrace{(Q^{-1}+H^TR^{-1}H)}_{=\Sigma^{-1}}s-2\underbrace{(\mu^TQ^{-1}+z^TR^TH)}_{=A^T}s\end{split}$$

We want to write this in the form $(s-m)^T\Sigma^{-1}(s-m)=s^T\Sigma^{-1}s-2m^T\Sigma^{-1}s+m^T\Sigma^{-1}m$ . Let $A^T=m^T\Sigma^{-1}$, then $\Sigma A=m$. So we add and subtract (subtraction is not shown because it is part of the normalizing constant) $A^T\Sigma A$ from the expression.

$$s^T\Sigma^{-1}s-2A^Ts+A^T\Sigma A=(s-\Sigma A)^T\Sigma^{-1}(s-\Sigma A)$$

Thus the posterior of $s$ follows a multivariate normal with mean $\Sigma A=(Q^{-1}+H^TR^{-1}H)^{-1}(Q^{-1}\mu+H^TRz)$ and variance covariance matrix $(Q^{-1}+H^TR^{-1}H)^{-1}$.