Law of total variance on conditional expectation

92 Views Asked by At

Let's take 3 random variables $X,Y,Z$ on the same probability space (or associated with the same experiment), then the law of total variance states that:

$$ V[X] = V[E[X|Z]] + E[V[X|Z]] $$

Then what if I want to apply the law of total variance on the following conditional variance: $V[X|Y]$? The goal is to decompose this conditional variance further. Would this result in something like $$ V[X|Y] = V[E[X|Z|Y]] + E[V[X|Z|Y]] $$ or maybe $$ V[X|Y] = V[E[X|Z,Y]|Y] + E[V[X|Z,Y]|Y] $$

Thanks a lot for any help or suggestions.

1

There are 1 best solutions below

2
On BEST ANSWER

$V\big[E[X\mid Z\mid Y]\big] + E\big[V[X\mid Z\mid Y]\big]$ does not make much sense notationally.

$V\big[E[X\mid Z]\mid Y\big] + E\big[V[X\mid Z]\mid Y\big]$ makes sense notationally but does not always give $V[X\mid Y]$.

$V[X\mid Y] = V\big[E[X\mid Z,Y]\mid Y\big] + E\big[V[X\mid Z,Y]\mid Y\big]$ both makes sense and is what you want.

To illustrate the distinction consider the example where $Y$ and $Z$ are i.i.d. Bernouilli$(\frac12)$, i.e. each independently $0$ or $1$ with equal probability and mean $\frac12$, and $X=Y+Z$:

  • You can find $V[X\mid Y]=V[Z]=\frac14$ since $Y$ and $Z$ are independent here.

  • $E[X \mid Z] =\frac12+Z$ and so $V\big[E[X\mid Z]\mid Y\big]=V\big[\frac12\mid Y\big]+V\big[Z\mid Y\big]=0+\frac14$, while $V[X \mid Z] =V[Y]=\frac14$ and so $E\big[V[X\mid Z]\mid Y\big]=\frac14$, giving $V\big[E[X\mid Z]\mid Y\big] + E\big[V[X\mid Z]\mid Y\big] = \frac12$, which is not $V[X\mid Y]$.

  • $E[X\mid Z,Y]=Y+Z$ and so $V\big[E[X\mid Z,Y]\mid Y\big]=V\big[Y\mid Y\big]+V\big[Z\mid Y\big]=0+\frac14$ again, while $V[X \mid Z,Y] =0$ (here, if you know $Z$ and $Y$ then you know $X$ exactly) and so $E\big[V[X\mid Z,Y]\mid Y\big]=0$, giving $V\big[E[X\mid Z,Y]\mid Y\big] + E\big[V[X\mid Z,Y]\mid Y\big] = \frac14$, as desired.