Product of marginal Gaussian and conditional Gaussian

Question

Suppose I have $$ \begin{align} p(x_1) &= N(x_1; 0, 1) \\ p(x_2 \mid x_1) &= N(x_2; x_1, 1) \end{align} $$ How do I compute $p(x_1 \mid x_2)$? I know how to compute their product, giving $N\left(\frac{x_1}{2}, \frac{1}{2}\right)$. The answer of the exercise is $N\left(\frac{x_2}{2}, \frac{1}{2}\right)$, so with $x_2$ rather than $x_1$ in the mean.

Answer 1

The following results work for multivariate Gaussians and can be used for univariate Gaussians as well. I came across them from one of the following texts (perhaps both):

1. Bayesian Filtering and Smoothing by Simo Särkkä
2. Linear Models in Statistics 2E by Rencher & Schaalje

Result #1: If random variables $x \in \mathbb R^n$ and $y \in \mathbb R^m$ have the Gaussian distributions $$x \sim \mathcal N (\mu, \Sigma)$$ $$y \,|\, x \sim \mathcal N (A x + b, \Omega)$$ then the joint distribution of $x, y$ $$\begin{pmatrix} x \\ y \end{pmatrix} \sim \mathcal{N} \left( \begin{pmatrix} \mu \\ A \mu + b \end{pmatrix}, \begin{pmatrix} \Sigma & \Sigma A^{\top} \\ A \Sigma & A \Sigma A^{\top} + \Omega \end{pmatrix} \right)$$

You can use result #1 to find the marginal distribution of $x_2$. Then you can use the definition of conditional probability to find the desired pdf: $$p(x_1 \,|\, x_2) = \frac{p (x_1, x_2)}{p (x_2)} = \frac{p(x_2 \,|\, x_1) p(x_1)}{p(x_2)}.$$

Using result #1 above, with $\mu = 0, \Sigma = 1, A = 1, b = 0, \Omega = 1$, we have that $$\begin{pmatrix} x_1 \\ x_2 \end{pmatrix} \sim \mathcal{N} \left( \begin{pmatrix} 0 \\ 0 \end{pmatrix}, \begin{pmatrix} 1 & 1 \\ 1 & 2 \end{pmatrix} \right).$$ Therefore, $x_2 \sim \mathcal{N} (0, 2)$. Using a bit of algebra, you'll find that $$p(x_1 \,|\, x_2) = \frac{ \mathcal{N} \left( x_2; x_1, 1 \right) \cdot \mathcal{N} \left( x_1; 0, 1 \right) }{ \mathcal{N} \left( x_2; 0, 2\right) } = \mathcal{N} \left( x_1; \frac{x_2}{2}, \frac12 \right)$$ which implies that $$x_1 \,|\, x_2 \sim \mathcal{N} \left( \frac{x_2}{2}, \frac12 \right).$$

---

Below is another result that can be used, along with result #1, to answer your question without working directly with pdfs.

Result #2: If random variables $x \in \mathbb R^n$ and $y \in \mathbb R^m$ have the joint Gaussian distribution \begin{equation} \begin{pmatrix} x \\ y \end{pmatrix} \sim \mathcal N \left( \begin{pmatrix} a \\ b \end{pmatrix}, \begin{pmatrix} A & C \\ C^{\top} & B \end{pmatrix} \right), \end{equation} then the conditional distributions of $x$ and $y$ are given as $$x \,|\, y \sim \mathcal N \left( a + C B^{-1} (y - b), A - C B^{-1} C^{\top} \right)$$ $$y \,|\, x \sim \mathcal N \left( b + C^{\top} A^{-1} (x - a), B - C^{\top} A^{-1} C \right)$$ provided $A$ and $B$ are invertible.

From result #1, we know that the joint distribution of $x_1$ and $x_2$ is $$\begin{pmatrix} x_1 \\ x_2 \end{pmatrix} \sim \mathcal{N} \left( \begin{pmatrix} 0 \\ 0 \end{pmatrix}, \begin{pmatrix} 1 & 1 \\ 1 & 2 \end{pmatrix} \right).$$ Using result #2, with $a=0, b=0, A=1, B=2, C=1$, we see that the conditional distribution of $x_1$ given $x_2$ is $$x_2 \,|\, x_1 \sim \mathcal{N} \left( \frac{x_2}{2}, \frac12 \right)$$