Conditional variance when there is uncertainity about the distribution of condition

80 Views Asked by At

Assume that $X \sim N(0,\sigma_x^2)$. Y has the following form \begin{align} Y &= \begin{cases} Y_1 \sim N(0, \sigma_1^2), & \text{w.p.} \quad \mu \\ Y_2 \sim N(0, \sigma_2^2), & \text{w.p.} \quad (1-\mu) \end{cases} \end{align} where $Cov(X,Y_1) =\Sigma_1$, and $Cov(X,Y_2) =\Sigma_2$. And also X and Y1, and X and Y2 are both bivariate normally distributed.

How can we calculate $V(X|Y=y)$? Is the following idea correct? \begin{align*} V(X|Y=y) = \mu V(X|Y=y_1)+(1-\mu)V(X|Y=y_2) \end{align*}

Basically, y is realized but it might be in the from of $y_1 \in Y_1$ or $y_2 \in Y_2$. What do you think?

1

There are 1 best solutions below

1
On BEST ANSWER

You can use the Conditional Variance Formula: $$\mathbb{V}(X \mid Y = y) = \mathbb{E}((X - \mathbb{E}(X\mid Y = y))^{2}\mid Y = y)$$ We need to calculate $\mathbb{E}(X \mid Y = y)$, and we can do this using a variation of the Law of Total Expectation. Let $A_i$ be the event $Y = Y_i$. $$\mathbb{E}(X \mid Y = y) = \int x\thinspace f_{x \mid y}dx = \int x\frac{f_{x,y}}{f_y}dx = \int x\sum_i \frac{f_{x,y,A_i}}{f_y}dx$$ $$= \int x\sum_i \frac{f_{x,y,A_i}}{f_{y,A_i}}\frac{f_{y,A_i}}{f_{y}}dx= \int x\sum_i \frac{f_{x,y,A_i}}{f_{y,A_i}}\frac{f_{y,A_i}}{f_{y}}dx = \sum_i\int x \thinspace f_{x \mid y, A_i}\mathbb{P}(A_i \mid Y = y)dx$$ $$= \sum_i \int x \thinspace f_{x \mid y, A_i}dx \cdot \mathbb{P}(A_i \mid Y = y) = \sum_i \mathbb{E}(X \mid Y = y, A_i)\mathbb{P}(A_i \mid Y = y).$$ So $$\mathbb{E}(X \mid Y = y) = \mathbb{E}(X \mid Y = y, Y = Y_1) \mathbb{P}(Y = Y_1 \mid Y = y) + \mathbb{E}(X \mid Y = y, Y = Y_2) \mathbb{P}(Y = Y_2 \mid Y = y).$$ We can use Bayes' Theorem to calculate the probabilities above: $$\mathbb{P}(Y = Y_1 \mid Y = y) = \frac{f_{Y_1}(y) \mathbb{P}(Y = Y_1)}{f_{Y_1}(y) \mathbb{P}(Y = Y_1) + f_{Y_2}(y) \mathbb{P}(Y = Y_2)}$$ where $$f_{Y_i}(y) = \frac{1}{\sqrt{2\pi \sigma_i^2}}e^{-y^2/2\sigma_i^2} \text{for } i = 1, 2.$$ So $$\mathbb{P}(Y = Y_1 \mid Y = y) = \frac{1/\sigma_1 e^{-y^2/2\sigma_1^2} \mu}{1/\sigma_1 e^{-y^2/2\sigma_1^2} \mu + 1/\sigma_2 e^{-y^2/2\sigma_2^2} (1 -\mu)} = \frac{\sigma_2 \mu e^{-y^2/2\sigma_1^2}}{\sigma_2 \mu e^{-y^2/2\sigma_1^2} + \sigma_1 (1 - \mu)e^{-y^2/2\sigma_2^2}}.$$ Also $$\mathbb{P}(Y = Y_2 \mid Y = y) = \frac{\sigma_1 (1 - \mu) e^{-y^2/2\sigma_2^2}}{\sigma_2 \mu e^{-y^2/2\sigma_1^2} + \sigma_1 (1 - \mu)e^{-y^2/2\sigma_2^2}}.$$ The conditional distribution of $X \mid Y = y, Y = Y_i$ is given by $$X \mid Y = y, Y = Y_i \sim N\left(\mu_x + \rho_i\frac{\sigma_x}{\sigma_i}(y - \mu_i), \sigma_x^2 (1 - \rho_i^2) \right)$$ where $\rho_i$ is the correlation of $X$ and $Y_i$, $\mu_x = \mathbb{E}(X) = 0$ and $\mu_i = \mathbb{E}(Y_i) = 0$.

Putting this together we have $$\mathbb{E}(X \mid Y = y) = \rho_1\frac{\sigma_x}{\sigma_1}y \cdot p_1 + \rho_2\frac{\sigma_x}{\sigma_2}y \cdot p_2 = \mu_{x \mid y},$$ where $p_i = \mathbb{P}(Y = Y_i \mid Y = y)$. Returning to the conditional variance formula, we now expand it, conditioning on $Y = Y_i$ again: $$\mathbb{V}(X \mid Y = y) = \mathbb{E}((X - \mu_{x\mid y})^{2}\mid Y = y)$$ $$= \mathbb{E}((X - \mu_{x\mid y})^{2}\mid Y = y, Y = Y_1)\mathbb{P}(Y = Y_1 \mid Y = y) + \mathbb{E}((X - \mu_{x\mid y})^{2}\mid Y = y, Y = Y_2)\mathbb{P}(Y = Y_2 \mid Y = y).$$ Now $$(X - \mu_{x\mid y})^{2} = (X - \mu_{x \mid y,i} + \mu_{x \mid y,i} - \mu_{x\mid y})^{2} = (X - \mu_{x \mid y,i})^2 + 2(X - \mu_{x \mid y,i})(\mu_{x \mid y,i} - \mu_{x\mid y}) + (\mu_{x \mid y,i} - \mu_{x\mid y})^2,$$ where $\mu_{x \mid y,i} = \mathbb{E}(X \mid Y = y, Y = Y_i)$, so $$\mathbb{E}((X - \mu_{x\mid y})^{2}\mid Y = y = Y_i) = \mathbb{E}((X - \mu_{x \mid y,i})^2 + 2(X - \mu_{x \mid y,i})(\mu_{x \mid y,i} - \mu_{x\mid y}) + (\mu_{x \mid y,i} - \mu_{x\mid y})^2\mid Y = y = Y_i)$$ $$= \sigma_x^2 (1 - \rho_i^2) + 0 + (\mu_{x \mid y,i} - \mu_{x\mid y})^2.$$ Therefore $$\mathbb{V}(X \mid Y = y) = \left(\sigma_x^2 (1 - \rho_1^2) + (\mu_{x \mid y,1} - \mu_{x\mid y})^2 \right)p_1 + \left(\sigma_x^2 (1 - \rho_2^2) + (\mu_{x \mid y,2} - \mu_{x\mid y})^2 \right)p_2.$$ $$ = \sigma_x^2 + \left(- \sigma_x^2 \rho_1^2 + (\mu_{x \mid y,1} - \mu_{x\mid y})^2 \right)p_1 + \left(- \sigma_x^2 \rho_2^2 + (\mu_{x \mid y,2} - \mu_{x\mid y})^2 \right)p_2.$$