Elegant geometric proof for this conditional expectation formula instead of brute force expansion

Question

I want to prove the following equation: Let $X$ be a real-valued RV with $E(X^2)<\infty$ and let $G, H\subseteq F$ be $\sigma$-algebras such that $H\subseteq G$. Then $$E\left((X-E(X\vert H))^2\right) = E\left((X-E(X\vert G))^2\right) + E\left((E(X\vert G)-E(X\vert H))^2\right)$$

The exercise says that instead of just using the definitions of (conditional) expectation and forcing the problem, there is an interesting geometric perspective contained in this equation which can also be used to prove it. I was wondering what it is.

This seems to resemble Pythagoras, but I can't close the connection. Does anyone see this geometric perspective? How is it connected to a proof?

Answer 1

One can interpret $E[X \mid H]$ as the $H$-measurable random variable $Y$ that minimizes the $L^2$ distance $E[(X-Y)^2]$. This can be viewed as an orthogonal projection of $X$ onto the space of $H$-measurable random variables. See the Wikipedia page for more info.

The geometric interpretation of your equation would indeed be a Pythagorean theorem involving a single projection onto $H$ (left-hand side) and two successive projections: first onto the larger sigma-algebra $G$, and then onto the smaller sigma-algebra $H$. Try drawing a picture of a projection of a 3-dimensional vector $x$ onto a line, along with the projection of $x$ onto a plane containing that line to visualize this.

In your algebraic proof, you probably used something like $$E[(X-E[X \mid G]) (E[X \mid G] - E[X \mid H])] = 0,$$ which geometrically is an orthogonality statement that gives you the right angle in the Pythagorean theorem.

Answer 2

Consider the vectors $\mathbf{a}=X-E[X|G]$ and $\mathbf{b}=E[X|G]-E[X|H]$ in $L^2(X)$. Since $E[X|G]$ is the orthogonal projection of $X$ onto the subspace of $G$-measurable functions we have that $\mathbf{a}$ is orthogonal to any $G$-measurable function, in particular it is orthogonal to $\mathbf{b}$. The pythogeran theorem for inner product spaces now says that $\|\mathbf{a}+\mathbf{b}\|^2=\|\mathbf{a}\|+\|\mathbf{b}\|^2$, i.e. your equation.