Proving $V(X) = E(V(X|Y)) + V(E(X|Y))$ using the pythagorean theorem

Question

I know the textbook proof of $$V(X) = E(V(X|Y)) + V(E(X|Y))$$ but I'm interested in understanding the weird proof/analogy with the pythagorean theorem my professor gave in class.

With $X, Y$ random variables, His explanation was that covariance defines an inner product, $X$ is a vector and $X|Y$ is the projection of $X$ on the "plane" of all posible functions of $Y$. We proved that Cov$(X - E(X|Y), G(Y)) = 0$ and that would mean that $X - E(X|Y)$ is normal to the "plane" of all possible functions of $Y$.

Fine so far but then he uses the fact that $X - E(X|Y)$ and $E(X|Y)$ are perpendicular to show that $$V(X) = V(X - E(X|Y)) + V(E(X|Y))$$ which is directly taken from the pythagorean theorem. My only problem with this is the way to get from $V(X - E(X|Y))$ to $E(V(X|Y))$. It's easy to show that $V(X - E(X|Y)) = E((X - E(X|Y))^2)$ which almost looks like the definition of variance for $X|Y$ but $X$ is not quite $X|Y$. What justifies that $$E((X - E(X|Y))^2) = E(E((X|Y - E(X|Y))^2)) = E(V(X|Y))$$? Is there something else I should be assuming about this to get to this conclusion?

PS: This is my first probability course. I did some linear algebra and I know inner products and abstract vector spaces but the professor didn't go as deep in this course. Thus I don't really know what formally are the components of $X$ as a vector and what is the "plane" of all possible functions of $Y$.

Answer 1

You have $ \mathbb V (X- \mathbb E[X|Y]) = \mathbb E[ (X-\mathbb E[X|Y])^2]$. The conditional variance is $ \mathbb V( A|B) := \mathbb E ( (A - \mathbb E [A|B])^2 | B ) $. what you need is the "tower rule/law of total expectation"[wiki] $$ \mathbb E\big[\mathbb E[A|B]\big] = \mathbb E [A]$$ which means that $$ \mathbb E[ (X-\mathbb E[X|Y])^2] = \mathbb E\left[ \mathbb E\big[ (X-\mathbb E[X|Y])^2\middle|Y\big]\right] = \mathbb E[\mathbb V(X|Y)]$$

I don't really know what formally are the components of $$ as a vector and what is the "plane" of all possible functions of $Y$.

$(a)$ Vectors doesn't have components until you choose a basis, so the next question is what is a basis of the space of finite variance random variables - I don't know of a "natural choice", but keep in mind that the answer can't be too simple since the space is infinite dimensional.

$(b)$ When you read "plane" replace with "linear vector space". This "plane" is the space of (finite variance) $\sigma(Y)$-measurable functions, i.e. the "space of functions depending on $Y$".