Normalizing factor for product of Gaussian densities - interpretation with Bayes theorem

1.4k Views Asked by At

The normalizing factor for the product of two multivariate Gaussian densities, $f(x)$ and $g(x)$ with mean vectors $a$ and $b$ respectively, and covariance matrices $A$ and $B$ respectively, is itself a Gaussian with exponent:

$-0.5(a-b)^T(A+B)^{-1}(a-b)$

i.e. the normalizing factor $c$ is:

$c=\cfrac{1}{\sqrt{2\pi\begin{vmatrix}A+B\end{vmatrix}}}e^{-\cfrac{1}{2}(a-b)^T(A+B)^{-1}(a-b)}$

At first sight, I'd hence interpret the normalizing factor as the value of a (hypothetical) density function with mean $b$ and covariance matrix $A+B$, evaluated at $a$.

Consider now an example, where $f(x)$ and $g(x)$ represent densities of two different statisticians' estimates for $x$. Statistician $A$ whose estimate has density $f(x)$ decides to update his estimate, after he saw the estimate for $x$ from the other statistician, named $B$: $x_b$.

$A's$ new estimate, i.e. an estimate conditional on $x_b$ has the density:

$f(x_a|x_b)$.

Now $B$ would in turn use $A's$ initial estimate to update her own estimate.

The new estimate of $B$, conditional on the estimate of $A$ would have the density:

$g(x_b|x_a)$.

Applying the Bayes rule for densities:

$g(x_b|x_a)=\cfrac{f(x_a|x_b)g(x_b)}{f(x_a)}$

So here, the normalizing factor is (also) $f(x_a)$ the unconditional (marginal) density of $x_a$.

Unfortunately, I cannot see the link between $-0.5(a-b)^T(A+B)^{-1}(a-b)$ and that marginal density. What am I missing?

P.S.: how to derive the expression above is indicated in the answer to the question given here: Product of two multivariate Gaussian pdfs - normalizing constant - where online references are given as well.