How to prove the Gaussian product theorem?

Question

Let $N\left(x ; \mu, P\right)$ denote the Gaussian distribution with expectation $\mu$ and covariance matrix $P$, then how can one show the following statement: $$N(x_2;Hx_1,P_2)N(x_1;\mu_1,P_1)＝N(x_2;H\mu_1,P_3)N(x_1;\mu,P)$$

where $$\begin{aligned} P_3 &=HP_{1} H^T＋P_{2}\\ \mu &=\mu_{1}+K\left(x_{2}-H \mu_{1}\right) \\ P &=(1-K H) P_{1}\\K&＝P_1H^TP_3^{-1}\end{aligned}$$ Can you give me some help? Any hint will be appreciated.

Answer 1

By expanding both sides and cancelling the same terms, we just need to show $$\frac{1}{|P_1P_2|^\frac12}e^{-\frac{(x_2-Hx_1)'P_2^{-1}(x_2-Hx_1)+(x_1-\mu_1)'P_1^{-1}(x_1-\mu_1)}{2}}=\frac{1}{|PP_3|^\frac12}e^{-\frac{(x_2-H\mu_1)'P_3^{-1}(x_2-H\mu_1)+(x_1-\mu)'P^{-1}(x_1-\mu)}{2}}$$ I will prove some formulars which will be used later.

1. \begin{equation}P^{-1}=H'P_2^{-1}H+P_1^{-1}\end{equation}

2. \begin{equation}KP_2=PH'\end{equation}

3. \begin{equation}K'P^{-1}K+P_3^{-1}=P_2^{-1}\end{equation}

Proof of 1.

\begin{align*} P(H'P_2^{-1}H+P_1^{-1})&=(1-KH)P_1(H'P_2^{-1}H+P_1^{-1})\\ &=1-KH+P_1H'P_2^{-1}H-KHP_1H'P_2^{-1}H\\ &=1-KH+P_1H'P_2^{-1}H-K(P_3-P_2)P_2^{-1}H\\ &=1+P_1H'P_2^{-1}H-P_1H'P_2^{-1}H\\ &=1 \end{align*} I have used the assumptions about $P_3$ and $K$ in the proof above.

Proof of 2.

Just need to show $P^{-1}K=H'P_2^{-1}$. \begin{align*} P^{-1}K&=(H'P_2^{-1}H+P_1^{-1})P_1H'P_3^{-1}\\ &=H'P_2^{-1}(P_3-P_2)P_3^{-1}+H'P_3^{-1}\\ &=H'P_2^{-1} \end{align*}

Proof of 3.

It's similar with 1. and 2.. Just use the assumptions.

Now let's prove \begin{aligned}&(x_2-Hx_1)'P_2^{-1}(x_2-Hx_1)+(x_1-\mu_1)'P_1^{-1}(x_1-\mu_1)\\=&(x_2-H\mu_1)'P_3^{-1}(x_2-H\mu_1)+(x_1-\mu)'P^{-1}(x_1-\mu)\end{aligned}

The left hand side equals to $x_1'(H'P_2^{-1}H+P_1^{-1})x_1-2x_1'(H'P_2^{-1}x_2+P_1^{-1}\mu_1)+x_2'P_2^{-1}x_2+\mu_1'P_1^{-1}\mu_1$. The right hand side equals to $$(x_1-\mu_1)'P^{-1}(x_1-\mu_1)+(H\mu_1-x_2)'(K'P^{-1}K+P_3^{-1})(H\mu_1-x_2)+2(H\mu_1-x_2)'K'P^{-1}(x_1-\mu_1)$$ which equals to $x_1'P^{-1}x_1-2x_1'\left[P^{-1}Kx_2+(P^{-1}-P^{-1}KH)\mu_1\right]+x_2'(K'P^{-1}K+P_3^{-1})X_2+\mu_1'\left[P^{-1}+H'(K'P^{-1}K+P_3^{-1})H-2H'K'P^{-1}\right]\mu_1+2x_2'\left[K'P^{-1}-(K'P^{-1}K+P_3^{-1})H\right]\mu_1$

Using the formulars we have proved above, the RHS equals to the LHS.

Now let's prove $$|P_1P_2|=|PP_3|$$ Firstly, $|PP_3|=|(1-KH)P_1P_3|$, so we just need to show $|(1-KH)P_3|=|P_2|$.

Using the famous conclusion $$|1-XY|=|1-YX|$$ We can obtain \begin{align*} |(1-KH)P_3|&=|(1-HK)P_3|\\ &=|P_3-HKP_3|\\ &=|P_3-HP_1H'|\\ &=|P_2| \end{align*} Now we have finished the whole proof.