Does $X-\operatorname{E}(X \mid Z)$ vary more or less then $X$?

Question

Does $X-\operatorname{E}(X \mid Z)$ vary more or less then $X$?

I've tried the following:

$$\begin{align} \operatorname{Var}(\color{green}{X-\operatorname{E}(X \mid Z)}) &= \operatorname{E}\left[\operatorname{Var}(\color{green}{X-\operatorname{E}(X \mid Z)} \mid Z)\right] + \operatorname{Var}\left[\operatorname{E}(\color{green}{X-\operatorname{E}(X\mid Z)}\mid Z)\right]\\ &= \operatorname{E}\left[\operatorname{Var}(X\mid Z)\right] + \operatorname{Var}\left[\operatorname{E}(X\mid Z)-\operatorname{E}\left\{\operatorname{E}(X\mid Z)\mid Z\right\}\right]\\ &= \operatorname{E}\left[\operatorname{Var}(X \mid Z)\right]\end{align}$$

I don't see how I can go on from here.

Answer 1

$\newcommand{\Var}{\operatorname{Var}} \newcommand{\Expect}{\operatorname{E}}$ The law of total variance tells us that, if $X$ and $Y$ are r.v.s on the same space, and $X$ has finite variance, then $$\Var(X) = \Expect\Var[X \mid Z]) + \Var(\Expect[X \mid Z]).$$ (We can assume that $X$ has finite variance, because if it doesn’t, then the answer is trivial.)

Proof is in the Wikipedia entry, so I won't repeat it here.

Dropping that into the equation above, we have $$ \begin{align*} \Var(X - \Expect[X \mid Z]) = \cdots &= \Expect(\Var[X \mid Z]) \\ &= \Var(X) - \Var(\Expect[X \mid Z]) \end{align*} $$ Since $\Expect[X \mid Z]$ is a random variable in its own right, it has non-negative variance. It follows that $$\Var(X - \Var[X \mid Z]) \leq \Var(X).$$ Further, we can deduce an equality condition: equality if and only if $\Var(\Expect[X \mid Z]) = 0$, which occurs if $X$ is a function of $Z$.

Answer 2

By Cauchy-Schwarz inequality, $$E(E(X\mid Z)^2)\geqslant E(E(X\mid Z))^2=E(X)^2,$$ hence $$E(X^2)-E(E(X\mid Z)^2)\leqslant E(X^2)-E(X)^2,$$ that is, $$\mathrm{Var}(X-E(X\mid Z))\leqslant \mathrm{Var}(X).$$