Finding $E[X\mid Y]$ and $\operatorname{Var}(X\mid Y)$ given mean and variance of $X$ and $Y$

Question

Suppose we have 2 normal distributions $X$ and $Y$ with mean $u_1$ and $u_2$ and variance $\sigma_1^2$ and $\sigma_2^2$; find $E[X\mid Y]$ and $\operatorname{Var}(X\mid Y)$.

I know $$E[X\mid Y] = \mu_1 + \rho\sigma_1 \frac{Y - u_2}{\sigma_2} $$ and $$\operatorname{Var}[X\mid Y] = \sigma_1 (1 - \rho^2)$$ but I can't prove it.

For $E[X\mid Y]$ I start with $$E[X\mid Y] = \int_{-\infty}^{+\infty} x f_{X|Y}(x\mid y)\ dx$$ but that doesn't work because for calculating $f_{X\mid Y}(x\mid y)$ I need $f_{X,Y}(x,y)$ I don't have that. Can any one help me?

Answer 1

The density approach will work. In the simplest case, assume that $X$ and $Y$ are each standard normal, with correlation $\rho$, so that the joint density of $(X,Y)$ is $$ f(x,y)=\frac1{2\pi\sqrt{1-\rho^2}}\exp \left[-\frac1{2(1-\rho^2)}(x^2-2\rho xy+y^2)\right] $$ while the marginal density of $Y$ is $$f(y)=\frac1{\sqrt{2\pi}}\exp\left[-\left(\frac{y^2}2\right)\right]. $$ The conditional density $\displaystyle f(x\mid y)=\frac{f(x,y)}{f(y)}$ is the ratio of these. So conditional on $Y=y$, the density of $X$ is $$\begin{align} f(x\mid y)&=\frac1{\sqrt{2\pi(1-\rho^2)}}\exp\left[-\frac1{2(1-\rho^2)}(x^2-2\rho xy+y^2-(1-\rho^2)y^2)\right]\\ &= \frac1{\sqrt{2\pi(1-\rho^2)}}\exp\left[-\frac1{2(1-\rho^2)}(x-\rho y)^2\right]\end{align} $$ which we recognize as the density of a normal random variable with mean $\rho y$ and variance $1-\rho^2$. It follows that $$ E(X\mid Y=y) = \rho y\qquad{\rm and}\qquad \operatorname{Var}(X\mid Y=y)=1-\rho^2.$$

---

For the general case, write $\displaystyle X':=\frac{X-\mu_1}{\sigma_1}$ and $\displaystyle Y':=\frac{Y-\mu_2}{\sigma_2}$. Apply the previous case to $X'$ and $Y'$, and conclude $$\begin{aligned} E\left (X\mid Y=y\right)&=E\left(\mu_1+\sigma_1 X'\biggm| Y'=\frac{y-\mu_2}{\sigma_2}\right)=\mu_1+\sigma_1 E\left(X'\biggm| Y'=\frac{y-\mu_2}{\sigma_2}\right)\\ &=\mu_1+\sigma_1\rho\left(\frac{y-\mu_2}{\sigma_2}\right) =\mu_1+\rho\frac{\sigma_1}{\sigma_2}(y-\mu_2) \end{aligned} $$ and $$\begin{aligned} \operatorname{Var}(X\mid Y=y)&=\operatorname{Var}\left(\mu_1+\sigma_1 X'\biggm| Y'=\frac{y-\mu_2}{\sigma_2}\right) =\sigma_1^2\operatorname{Var}\left( X'\biggm| Y'=\frac{y-\mu_2}{\sigma_2}\right)\\ &=\sigma_1^2(1-\rho^2).\end{aligned} $$

Answer 2

Let us take for granted that $$\frac{X-\mu_1}{\sigma_1} = \rho \frac{Y-\mu_2}{\sigma_2} + \sqrt{1-\rho^2} Z \tag{$*$}$$ where $Z \sim N(0,1)$ is independent of $Y$. See the end of my answer for an explanation.

Then \begin{align} E[X \mid Y] &= \mu_1 + \sigma_1 E[(X-\mu_1)/\sigma_1 \mid Y] \\ &= \mu_1 + \sigma_1 \left( E[\rho (Y-\mu_2)/\sigma_2 \mid Y] + E[\sqrt{1-\rho^2} Z \mid Y] \right) \\ &= \mu_1 + \sigma_1 (\rho(Y - \mu_2) / \sigma_2) + \sqrt{1-\rho^2} E[Z] \\ &= \mu_1 + \rho \frac{\sigma_1}{\sigma_2} (Y-\mu_2). \end{align} Note that Ottavio Bartenor fixed a typo in your original expression for $E[X\mid Y]$.

Similarly, \begin{align} \text{Var}(X \mid Y) &= \sigma_1^2 \text{Var}((X-\mu_1)/\sigma_1 \mid Y) \\ &= \sigma_1^2 \text{Var}(\rho(Y-\mu_2)/\sigma_2 + \sqrt{1-\rho^2} Z \mid Y) \\ &= \sigma_1^2 \text{Var}(\sqrt{1-\rho^2} Z \mid Y) \\ &= \sigma_1^2 (1-\rho^2) \text{Var}(Z) \\ &= \sigma_1^2(1-\rho^2). \end{align} Note that the expression in your post has a typo.

---

Suppose $Y \sim N(\mu_2, \sigma_2^2)$ and $Z \sim N(0,1)$ are independent. Let $X$ satisfy the above equality ($*$). The claim is that $(X,Y)$ follows a bivariate normal distribution with parameters $\mu_1, \sigma_1, \mu_2 ,\sigma_2, \rho$.

You can check that $X$ has mean $\mu_1$ and variance $\sigma_1^2$. You can also check that the correlation between $X$ and $Y$ is $\rho$. You can also check that the marginal distribution of $X$ is normal, because it is a linear combination of independent normal random variables. Finally, to justify that $(X,Y)$ is jointly (bivariate) normal, you can appeal to an equivalent characterization of jointly normal distributions by noting that any linear combination of $X$ and $Y$ is normal because it can be written as a linear combination of the independent normal random variables $Y$ and $Z$