What is the product of these two Gaussian PDFs

Question

I am given two random variables $x,y$ where \begin{align*} x &\sim \mathcal{N}(0,I) & \text{where}\ x \in \mathbb{R}^d \\ y|x &\sim \mathcal{N}(Ax+b,a^2I) & \text{where} \ y \in \mathbb{R}^n. \end{align*}

for some matrix $A$ and real number $a$. I want to know the general fact that I can say about the distribution of $(x,y)$. Certainly its given by the product of the distribution of $x$ and $y|x$. However, that product is messy and I am pretty sure the result of the joint distribution should be a Gaussian; however, I'm not sure what the parameters are.

I would also like to know the general rule when you multiply the pdf of two Gaussians (if I am correct that there is such a rule).

Answer 1

We realize the distribution of $Y$ as follows:

Let $X \sim \mathcal{N}(0, \mathbf{I}_m)$ and $Z \sim \mathcal{N}(0, \mathbf{I}_n)$ be independent, and set

$$ Y = AX + aZ + b. $$

Then the conditional distribution of $Y$, given that $X = x$, is precisely $\mathcal{N}(Ax + b, a^2\mathbf{I}_n)$. So, $Y$ has the desired distribution. Now, since $Y$ is a linear combination of independent gaussian random variables, it is also gaussian and hence its distribution is completely characterized by the mean and covariance matrix. However,

$$ \mathbf{E}[Y] = b \qquad\text{and}\qquad \mathbf{Var}(Y) = AA^{\top} + a^2 \mathbf{I}_n. $$

So it follows that

$$ Y \sim \mathcal{N}(b, AA^{\top} + a^2 \mathbf{I}_n). \tag{*} $$

---

Addendum. You can also get this result directly from the computation. Indeed,

\begin{align*} p(y) &= \int_{\mathbb{R}^m} p(y|x)p(x) \, \mathrm{d}x \\ &\propto \int_{\mathbb{R}^m} \exp\left( - \frac{\|y - Ax - b\|^2}{2a^2} - \frac{\|x\|^2}{2} \right) \, \mathrm{d}x. \end{align*}

Now we focus on the exponent of the integrand. Completing the square,

\begin{align*} &- \frac{\|y - Ax - b\|^2}{2a^2} - \frac{\|x\|^2}{2} \\ &\quad = - \frac{1}{2a^2}\left( \langle x, (A^{\top}A + a^2 \mathbf{I}_m) x\rangle - 2 \langle x, A^{\top}(y - b)\rangle + \|y - b\|^2 \right) \\ &\quad = - \frac{1}{2a^2}\left( \| B x - B^{-1} A^{\top}(y - b) \|^2 + \langle y - b, a^2\Sigma^{-1} (y - b) \rangle \right), \end{align*}

where $B = \sqrt{A^{\top}A + a^2 \mathbf{I}_m}$ and $\Sigma = AA^{\top} + a^2\mathbf{I}_n$, and we utilized the identity

\begin{align*} \mathbf{I}_n - (B^{-1}A^{\top})^{\top}(B^{-1}A^{\top}) &= \mathbf{I}_n - A(A^{\top}A + a^2\mathbf{I}_m)^{-1}A^{\top} \\ &= a^2(AA^{\top} + a^2\mathbf{I}_n)^{-1} \\ &= a^2\Sigma^{-1}. \end{align*}

Plugging this back and integrating out the resulting gaussian density with respect to $x$, we are left with

\begin{align*} p(y) &\propto \exp\left( - \frac{\langle y - b, \Sigma^{-1} (y - b) \rangle}{2} \right), \end{align*}

which is equivalent to $\text{(*)}$.

Answer 2

According to your question, $y|x$ has mean $Ax+b$, so $E[y|x]=E[Ax+b]=AE[x]+b=b$, and variance of $E[Y]=AA^T+a^2\mathcal{I}$ so your normal distribution will be $\mathcal{N}(b,a^2\mathcal{I})$ as Y is a Gaussian variable because it is a linear combination of two Gaussians random variables.