Show that the variance, $\mathbb E((x-\mathbb E(x))^2)$, can be written as $\mathbb E(x^2)-(\mathbb E(x))^2$

Question

This question has been set in the Christmas work for the chemists at oxford uni and the hint that was given in the problem sheet was "does $\mathbb E(x)$ depend on $x$?".

There is a derivation on Wikipedia: $$\mathbb E((x-\mathbb E(x))^2)$$ Expand the brackets (I get this bit) $$\mathbb E(x^2-2x\mathbb E(x)+\mathbb E(x)^2)$$ Use the fact the expectation value of the sum of the terms is the same as the sum of the expectation values of the terms: $$\mathbb E(x^2)-2\mathbb E(x\mathbb E(x))+\mathbb E(\mathbb E(x)^2)$$ This step isn't written explicitly in the derivation on Wikipedia but it must be the case. Now the step that I don't understand: $$\mathbb E(x^2)-2\mathbb E(x)\mathbb E(x)+\mathbb E(\mathbb E(x)^2)$$ Why does the term in the middle not change to $2\mathbb E(x)\mathbb E(\mathbb E(x))$?

Then collect the terms:$$\mathbb E(x^2)-\mathbb E(x)^2$$

Also, this doesn't seem to utilise the hint given by my lecturer. Is there a better way of doing it?

Answer 1

Expected value of a constant is a constant:

$$\begin{align} E(a) &= \int_{-\infty}^{\infty}af(x)dx \\ &= a\int_{-\infty}^{\infty}f(x)dx \\ &= a \end{align}$$ since $\int_{-\infty}^{\infty}f(x)dx = 1$ the total probability. Hence, $$\begin{align} E\left((x-E(x))^2\right) &= E\left(x^2 -2xE(x) + E(x)^2\right) \\ &= E(x^2) -2E(x)E(x) +E(x)^2 \\ &= E(x^2) - E(x)^2 \end{align}$$

Answer 2

The hint is to be understood as $\mathbb E(x)$ only depends on the distribution of $x$, but not on the samples, thus an equivalent formula to that statement would be $$\mathbb E(\mathbb E(x)) = \mathbb E(x)$$

Answer 3

Lets set $E[X]=\mu=\int_{-\infty}^{\infty}xf(x)dx$

$$\text{Var}(X) = E[(X-\mu)^2] = E[X^2-2\mu X+\mu^2]$$

Now calculate this expectation:

$$\begin{align} E[X^2-2\mu X+\mu^2] &= \int_{-\infty}^{\infty} (x^2-2\mu x+\mu^2)f(x)dx \\ &=\int_{-\infty}^{\infty}x^2f(x)dx - 2\mu\int_{-\infty}^{\infty}xf(x)dx + \mu^2\int_{-\infty}^{\infty}f(x)dx \\ &= E[X^2]-2\mu\mu+\mu^2 \\ &= E[X^2]-\mu^2 \end{align}$$

For more details on the subject consult Sheldon Ross Introduction to Probability Models.