Expectation expression

Question

What is the difference between $E[X^2]$ and $E^2[X]$?

Consider $X$ is the number of heads when a fair coin is tossed twice.

Please explain clearly in general and w.r.t this example also.

Answer 1

$E(X^2)=\sum_{i=1}^{n} x_{i}^{2} P(x_i)$ and $(E(X))^2= \left(\sum_{i=1}^{n} x_{i} P(x_i)\right)^2$ when the support of X is $x_1, x_2, ..., x_n$. On the other hand, the first one is the Mean Square but the second is Square of Mean. For tossing a fair coin twice, $E(X^2)=3/2$ but $\left(E(X)\right)^2= (1)^2=1$.

Answer 2

Consider the random variable given by $X_0 = X - \def\exp{\mathbb{E}}\exp[X]$. Let me call this random variable the deviation of $X$ (from its expectation value).

The key observation is that the expectation of the deviation of $X$ is always $0$, no matter what $X$ is:

$$\exp[X_0] = \exp[X] - \exp[\exp[X]] = \exp[X] - \exp[X] = 0.$$

In contrast, the square of the deviation of $X$, namely $X_0^2$, is always non-negative, and therefore $\exp[X_0^2] > 0$.¹ I'll call this random variable $X_0^2$ the squared deviation of $X$.

It's not difficult to derive an expression for $\exp[X_0^2]$ in terms of $X$. First, expand $X_0^2$:

$$X_0^2 = (X - \exp[X])^2 = X^2 - 2 X\cdot\exp[X] + \exp[X]^2\;.$$

Now, take expectations on both sides of $=$, and apply the linearity of expectation to the RHS:

$$ \begin{array}{rcl} \exp[X_0^2] & = & \exp[X^2] - 2\exp[X]\cdot\exp[X] + \exp[\exp[X]^2] \\ & = & \exp[X^2] - 2\exp[X]^2 + \exp[X]^2 \\ & = & \exp[X^2] - \exp[X]^2 \end{array} $$

(In going from the first to the second equality, I used the fact that $\exp[\exp[X]^2] = \exp[X]^2$, since $\exp[X]^2$ is a constant for any fixed random variable $X$.)

Note that the expression on the last line of the derivation above is precisely the difference $\exp[X^2] - \exp[X]^2$. IOW, the derivation shows that this difference is nothing more than the expectation of the squared deviation of $X$.

This realization makes it particularly easy to see what the value of this difference would be when $X$ is the number of heads out of two tosses of a fair coin. In this case, the deviation $X_0$ can be either $-1, 0,$ or $1$, and therefore, the squared deviation $X_0^2$ can be either $0$ or $1$. Only the latter case contributes to the expectation of $X_0^2$. It corresponds to the event where both flips turn out heads or both turn out tails. This event has probability $\frac{1}{4} + \frac{1}{4} = \frac{1}{2}$. Therefore, for this $X$, the expectation of the squared deviation is $1 \cdot \frac{1}{2} = \frac{1}{2}$.

_{¹ To be more precise, $\exp[X_0^2] \geq 0$, but if $\mathbb{P}[X \neq \exp[X]] > 1$, then $\exp[X_0^2] > 0$.}

Answer 3

What is the difference between $E[X^2]$ and $E^2[X]$?

Answer:

• The variance of $X$.