How can we compute this conditional expectation?

137 Views Asked by At

The problem:

Suppose $(X_1,...,X_d) \in \mathbb{R}^d$ and $Y \in \mathbb{R}$ are random variable such that $(X_1,...,X_d,Y)$ is a gaussian vector.

How can we prove $\mathbb{E}[Y\mid \sigma(X)]$ is a linear combination of $(X_i)$?

I tried first proving $\operatorname{Span}_{\mathbb{R}} X_i$ is dense in $L^2(\Omega,\sigma(X))$ or equal but I cannot conclude.

Edit:

As remarked in the comments it would be more interesting if we consider $\operatorname{Span}_{\mathbb{R}} (X_i,1)$.

2

There are 2 best solutions below

1
On

Suppose $f_{X,Y}$ is the joint density of the variables with $Y$ included. This is a Gaussian density function by assumption. The conditional mean is almost surely equal to the mean of the conditional density

$$ f_{Y|X}(y|x) = \frac{f_{X,Y}(x,y)}{\int_{\mathbb{R}}f_{X,Y}(x,y)dy}. $$ Namely, you can show that $$ E(Y|X) = \int_{\mathbb{R}} yf_{Y|X}(y|X)dy \;\; a.s. $$

The later term you can calculate, and it is linear in $X$, as desired.

1
On

Let $\mu_Y:=\mathsf{E}Y$, $\mu_X:=\mathsf{E}X$, $\Sigma_X:=\operatorname{Var}(X)$ $\Sigma_Y:=\operatorname{Var}(Y)$, and $\Sigma_{X,Y}=\mathsf{E}X(Y-\mu_Y)$. Consider the following transformation (assuming that $\Sigma_X$ is invertible): $$ Z:=Y-\Sigma_{X,Y}^{\top}\Sigma_X^{-1}X. $$ Since \begin{align} \mathsf{E}X(Z-\mathsf{E}Z)^{\top}&=\mathsf{E}X(Y-\mu_Y-\Sigma_{X,Y}^{\top}\Sigma_X^{-1}(X-\mu_X))^{\top} \\ &=\Sigma_{X,Y}-\Sigma_X\Sigma_X^{-1}\Sigma_{X,Y}=0, \end{align} $Z$ and $X$ are independent (they're jointly Gaussian). Then \begin{align} \mathsf{E}[Y\mid X]&=\mathsf{E}[Z\mid X]+\mathsf{E}[\Sigma_{X,Y}^{\top}\Sigma_X^{-1}X\mid X] \\ &=\mathsf{E}Z+\Sigma_{X,Y}^{\top}\Sigma_X^{-1}X=\mu_Y+\Sigma_{X,Y}^{\top}\Sigma_X^{-1}(X-\mu_X) \quad\text{a.s.} \end{align}


When $\Sigma_X$ is singular, some coordinates of $X$ are affine functions of a subset $X^{*}$ of dimension $\operatorname{rank}(\Sigma_X)$ (assuming that $\operatorname{rank}(\Sigma_X)\ge 1$). In that case $\mathsf{E}[Y\mid X]=\mathsf{E}[Y\mid X^*]$, and $\operatorname{Var}(X^*)$ is invertible.