How to justify that $\text{Var}\Big(\frac{1}{n}\sum_{i = 1}^nX_i\Big) = \frac{1}{n^2}\sum_{i = 1}^n\text{Var}(X_i)$

Question

In the sum of uncorrelated variables (Bienaymé formula), it is stated that $$ \begin{align*} \text{Var}\Big(\bar{X}\Big) &= \text{Var}\Big(\frac{1}{n}\sum_{i = 1}^nX_i\Big)\tag{1}\\ &= \frac{1}{n^2}\sum_{i = 1}^n\text{Var}(X_i)\tag{2}\\ &= \frac{1}{n^2}\sigma^2\\ &=\frac{\sigma^2}{n} \end{align*} $$

How do we justify the changes going from (1) to (2)?

Answer 1

$\begin{align*}\text{Var}\Big(\frac{1}{n}\sum_{i = 1}^nX_i\Big) &= \mathbb{E}\left(\left(\frac{1}{n}\sum_{i = 1}^nX_i\right)^2\right) - \left(\mathbb{E}\left(\frac{1}{n}\sum_{i = 1}^nX_i\right)\right)^2\\ &= \frac{1}{n^2}\left(\mathbb{E}\left(\left(\sum_{i = 1}^nX_i\right)^2\right) - \left(\mathbb{E}\left(\sum_{i = 1}^nX_i\right)\right)^2\right)\\ &=\frac{1}{n^2}\sum_{i = 1}^n\text{Var}(X_i), \end{align*}$

where in the second equality we used the fact that $\mathbb{E}$ is linear and for the third equality you need to use that $X_i's$ are uncorrelated. More precisely, you should use the following equality:

$Var(X + X^{\prime}) = Var(X) + Var(X^{\prime}) + 2Cov(X, X^{\prime}) = Var(X) + Var(X^{\prime})$,

when $X$ and $X^{\prime}$ are uncorrelated.

Answer 2

Well $\operatorname{Var}(cY)=c^2\operatorname{Var}(Y)$ is true for any random variable with second moment and any $c$.

Furthermore, we see that if $Y$ and $Z$ have second moment, then, by bi-linearity of the covariance, \begin{align} \operatorname{Var} (Z+Y)&=\operatorname{Cov}(Z+Y,Z+Y)\\ &=\operatorname{Cov}(Z,Z)+\operatorname{Cov}(Y,Z)+\operatorname{Cov}(Z,Y)+\operatorname{Cov}(Y,Y)\\ &=\operatorname{Var(Z)}+\operatorname{Var(Z)}+2\operatorname{Cov}(Z,Y) \end{align}

Hence, if $\operatorname{Cov}(Z,Y)=0$, then $\operatorname{Var}(Z+Y)=\operatorname{Var}(Z)+\operatorname{Var}(Y)$. You can proceed by induction if you wish.