How do I prove the formula for the product of two Gaussians?

57 Views Asked by At

I need to prove that the product of two multivariate Gaussian densities $\mathcal{N}(\textbf{x}|\textbf{a}, \textbf{A})$ $\mathcal{N}(\textbf{x}|\textbf{b}, \textbf{B})$ is an unnormalized Gaussian distribution $c\mathcal{N}(\textbf{x}|\textbf{c}, \textbf{C})$ with

$\textbf{C} = (\textbf{A}^{-1} + \textbf{B}^{-1})^{-1}$

$\textbf{c} = \textbf{C}(\textbf{A}^{-1}\textbf{a} + \textbf{B}^{-1}\textbf{b})$

$c = (2\pi)^{-\frac{D}{2}} |\textbf{A} + \textbf{B}|^{-\frac{1}{2}} exp\left(-\frac{1}{2}(\textbf{a}-\textbf{b})^{T}(\textbf{A}+\textbf{B})^{-1}(\textbf{a}-\textbf{b})\right)$

Here's how far I got:

$\mathcal{N}(\textbf{x}|\textbf{a}, \textbf{A}) = (2\pi)^{-\frac{D}{2}} |\textbf{A}|^{-\frac{1}{2}} exp[-\frac{1}{2}(\textbf{x}-\textbf{a})^{T}\textbf{A}^{-1}(\textbf{x}-\textbf{a})]$

$\mathcal{N}(\textbf{x}|\textbf{b}, \textbf{B}) = (2\pi)^{-\frac{D}{2}} |\textbf{B}|^{-\frac{1}{2}} exp[-\frac{1}{2}(\textbf{x}-\textbf{b})^{T}\textbf{B}^{-1}(\textbf{x}-\textbf{b})]$

$\mathcal{N}(\textbf{x}|\textbf{a}, \textbf{A})\mathcal{N}(\textbf{x}|\textbf{b}, \textbf{B}) = \left((2\pi)^{-\frac{D}{2}} |\textbf{A}|^{-\frac{1}{2}} exp[-\frac{1}{2}(\textbf{x}-\textbf{a})^{T}\textbf{A}^{-1}(\textbf{x}-\textbf{a})]\right) \left((2\pi)^{-\frac{D}{2}} |\textbf{B}|^{-\frac{1}{2}} exp[-\frac{1}{2}(\textbf{x}-\textbf{b})^{T}\textbf{B}^{-1}(\textbf{x}-\textbf{b})]\right)$

Taking the log of this expression, we get

$\ln\left((2\pi)^{-\frac{D}{2}} |\textbf{A}|^{-\frac{1}{2}} exp[-\frac{1}{2}(\textbf{x}-\textbf{a})^{T}\textbf{A}^{-1}(\textbf{x}-\textbf{a})](2\pi)^{-\frac{D}{2}} |\textbf{B}|^{-\frac{1}{2}} exp[-\frac{1}{2}(\textbf{x}-\textbf{b})^{T}\textbf{B}^{-1}(\textbf{x}-\textbf{b})]\right)$

$ = \ln((2\pi)^{-D}|\textbf{A}|^{-\frac{1}{2}}|\textbf{B}|^{-\frac{1}{2}})+(-\frac{1}{2}(\textbf{x}-\textbf{a})^{T}\textbf{A}^{-1}(\textbf{x}-\textbf{a})-\frac{1}{2}(\textbf{x}-\textbf{b})^{T}\textbf{B}^{-1}(\textbf{x}-\textbf{b}))$

$ = \ln((2\pi)^{-D}|\textbf{A}|^{-\frac{1}{2}}|\textbf{B}|^{-\frac{1}{2}})-\frac{1}{2}((\textbf{x}-\textbf{a})^{T}\textbf{A}^{-1}(\textbf{x}-\textbf{a})+(\textbf{x}-\textbf{b})^{T}\textbf{B}^{-1}(\textbf{x}-\textbf{b}))$

$ = \ln((2\pi)^{-D}|\textbf{A}|^{-\frac{1}{2}}|\textbf{B}|^{-\frac{1}{2}})-\frac{1}{2}(\textbf{x}^{T}\textbf{A}^{-1}\textbf{x}-\textbf{a}^{T}\textbf{A}^{-1}\textbf{x}-\textbf{x}^{T}\textbf{A}^{-1}\textbf{a}+\textbf{a}^{T}\textbf{A}^{-1}\textbf{a}+\textbf{x}^{T}\textbf{B}^{-1}\textbf{x}-\textbf{b}^{T}\textbf{B}^{-1}\textbf{x}-\textbf{x}^{T}\textbf{B}^{-1}\textbf{b}+\textbf{b}^{T}\textbf{B}^{-1}\textbf{b})$

$ = \ln((2\pi)^{-D}|\textbf{A}|^{-\frac{1}{2}}|\textbf{B}|^{-\frac{1}{2}})-\frac{1}{2}(\textbf{x}^{T}(\textbf{A}^{-1}+\textbf{B}^{-1})\textbf{x}-\textbf{x}^{T}(\textbf{A}^{-1}\textbf{a}+\textbf{B}^{-1}\textbf{b})-(\textbf{a}^{T}\textbf{A}^{-1}+\textbf{b}^{T}\textbf{B}^{-1})\textbf{x}+\textbf{a}^{T}\textbf{A}^{-1}\textbf{a}+\textbf{b}^{T}\textbf{B}^{-1}\textbf{b})$

$(\textbf{a}^{T}\textbf{A}^{-1}+\textbf{b}^{T}\textbf{B}^{-1})\textbf{x}$ is a scalar, so it is equal to its own transpose.

$\Longrightarrow (\textbf{a}^{T}\textbf{A}^{-1}+\textbf{b}^{T}\textbf{B}^{-1})\textbf{x} = ((\textbf{a}^{T}\textbf{A}^{-1}+\textbf{b}^{T}\textbf{B}^{-1})\textbf{x})^{T} = \textbf{x}^{T}(\textbf{a}^{T}\textbf{A}^{-1}+\textbf{b}^{T}\textbf{B}^{-1})^{T}$

$ = \textbf{x}^{T}(\textbf{A}^{-1}\textbf{a}+\textbf{B}^{-1}\textbf{b})$ inverse of a symmertric matrix is also symmetric.

Substituting, we get

$ = \ln((2\pi)^{-D}|\textbf{A}|^{-\frac{1}{2}}|\textbf{B}|^{-\frac{1}{2}})-\frac{1}{2}(\textbf{x}^{T}(\textbf{A}^{-1}+\textbf{B}^{-1})\textbf{x}-2\textbf{x}^{T}(\textbf{A}^{-1}\textbf{a}+\textbf{B}^{-1}\textbf{b})+\textbf{a}^{T}\textbf{A}^{-1}\textbf{a}+\textbf{b}^{T}\textbf{B}^{-1}\textbf{b})$

Substituting the values of $\textbf{C}$ and $\textbf{c}$, the above expression becomes,

$ = \ln((2\pi)^{-D}|\textbf{A}|^{-\frac{1}{2}}|\textbf{B}|^{-\frac{1}{2}})-\frac{1}{2}(\textbf{x}^{T}\textbf{C}^{-1}\textbf{x}-2\textbf{x}^{T}\textbf{C}^{-1}\textbf{c}+\textbf{a}^{T}\textbf{A}^{-1}\textbf{a}+\textbf{b}^{T}\textbf{B}^{-1}\textbf{b})$

At this point, I am stuck, What do I do next?