Variance of a random variable in terms of expected value?

Question

When I first encountered the variance of a random variable, I found it in the form: $$\text{Var}(X) = \sum_{i=1}^n (\mu - x_i)^2p_i$$ which is pretty intuitive: it's the sum of each squared distance from the mean times its respective probability of happening, and pairs nicely with $E(X) = \sum_{i=1}^n x_i p_i$. However, in this answer I saw a proof that used variance in this form: $$\text{Var}(X) = E[(X-E(X))^2]$$ which I can't seem to derive from the first formula. How is this second definition of variance proved?

Answer 1

Three key realizations play a role here:

• $\displaystyle \mu := \mathbf{E}[X] := \sum_i x_i p_i$
• $\displaystyle \sum_i p_i = 1$
• $\displaystyle \mathbf{E}[f(X)] = \sum_i f(x_i) p_i$

Then we have

$$\begin{align*} \text{var}(X) &:= \sum_i \left( x_i - \mu \right)^2 p_i \\ &= \sum_i x_i^2 p_i - 2 x_i \mu p_i + \mu^2 p_i \\ &= \sum_i x_i^2 p_i - 2 \mu \sum_i x_i p_i + \mu^2 \sum_i p_i \\ &= \mathbf{E}[X^2] - 2 \mu \cdot \mathbf{E}[X] + \mu^2 \\ &= \mathbf{E}[X^2] - 2 \cdot \mathbf{E}[X]^2 + \mathbf{E}[X]^2 \\ &= \mathbf{E}[X^2] - \mathbf{E}[X]^2 \tag{$\ast$} \\ &= \sum_i x_i^2 p_i - \left( \sum_i x_i p_i \right)^2 \\ &= \sum_i x_i^2 p_i - \sum_{i,j} x_i x_j p_i p_j \\ &= \sum_i x_i p_i \left( x_i - \sum_{j} x_j p_j \right) \\ &= \sum_i x_i \left( x_i - \mathbf{E}[X] \right) p_i \\ &= \mathbf{E} \big[ X - \mathbf{E}[X] \big] \end{align*}$$

Note also that $(\ast)$ gives us another common formulation for variance of a random variable.

Answer 2

Using the Law of the Unconscious Statistician, if $t(\cdot)$ is a function and $X$ is a real-valued random variable with pdf $f$, $$ \mathbb{E}[t(X)] = \int_\mathbb{R} t(x) f(x) dx, $$ which for a discrete distribution with support in $\{x_i\}_{i=1}^n$ with probabilities $f(x_i) = p_i$ and $\sum_{i=1}^n p_i = 1$, we have $$ \mathbb{E}[t(X)] = \sum_{i=1}^n t(x_i) p_i, $$ which you can now apply to the function $t(X) = (X- \mu)^2$, where $\mu$ is the expected value of $X$, which is a constant.

The RHS will be identical to your first formula's RHS and the LHS will be the second formula's RHS.

Answer 3

For anybody who wants an intuitive explanation to supplement the more rigorous proofs:

1. Let $X$ be the random variable you want to take the variance of.
2. Let us define another random variable $Y = (X-E(X))^2$.
3. $Y$ is the distribution of squared distances from $E(X)$.
4. Therefore, $E(Y)$ is the average squared distance from $E(X)$.
5. $\text{Var}(X)$ is defined as the average squared distance from the $E(X)$.
6. Therefore, $\text{Var}(X) = E(Y)$.
7. Therefore, $\text{Var}(X) = E[(X-E(X))^2]$.