To begin with, the standard iterated law of probability is as follows.
$$ \mathbb E X = \mathbb E [\mathbb E(X|Y)]. (1) $$
I am perfectly happy with $(1)$ and there is also some quite good discussion on the intuition here. However, the extension of this property is more troublesome to me. It states that
$$ \mathbb E (X|Y) = \mathbb E [\mathbb E(X|Y, Z)|Y]. (2) $$
I found a proof here which basically just restated the definition. I still do not get it mathematically or intuitively.
- Could anyone provide a/an concrete/intuitive example to explain how $(2)$ works, please? How is $(2)$ an extension of $(1)$, please? Is there a more straightforward way to prove it, please? Thank you!
In addition, by letting $X$ equal to $X|Y$ and $Y$ equal to $Z$ in $(1)$, one has
$$ \mathbb E(X|Y) = \mathbb E\{\mathbb E[(X|Y)|Z]\}. (3) $$
- What does $(3)$ mean exactly, please? Is $(3)$ equal to $(2)$, please? Thank you!
You should view $E[X|Y]$ as $E[X|\sigma_{Y}]$ where $\sigma_{Y}$ is the $\sigma$-algebra generated by the random variable $Y$. It stands to reason that $\sigma_{Y}$ is a sub-$\sigma$-algebra of the $\sigma$-algebra generated by $Y$ and $Z$. Loosely speaking, the information that you get from knowing $Y$ alone should be a "subset" of the information that you get from knowing $Y$ and $Z$ together.
The iterated conditioning law says that if $F$ is a sub-$\sigma$-algebra of $G$, then $$ E[E[X|G]|F] = E[X|F] $$
You can take it from here on I think.