Variance of the sum of uncorrelated variables

Question

In using the Bienaymé formula to find the variance of means, I do not understand why

$$\operatorname{Var}\!\left(\frac{1}{n}\sum_{i=1}^{n}{X_i}\right)=\frac{1}{n^2}\sum_{i=1}^n{\operatorname{Var}(X_i)}$$

I assume it is a matter of simple algebraic manipulation, but I do not understand it. Could someone please explain this property?

Answer 1

Here is a detailed derivation.

So, let's start with the "easy" part: $$ \operatorname{Var} \left(\frac{1}{n}\sum_{i=1}^n X_i\right) = \frac{1}{n^2}\operatorname{Var} \left(\sum_{i=1}^n X_i\right)\,. $$ This is just because $\operatorname{Var} ( a X) = a^2 \operatorname{Var} X$ for any constant $a$ (and r.v. $X$ such that the variance exists).

Now, the "hard" part, where the non-correlation will be used: $$\begin{align*} \operatorname{Var} \left(\sum_{i=1}^n X_i\right) &= \mathbb{E}\left[\left(\sum_{i=1}^n X_i-\sum_{i=1}^n \mathbb{E}[X_i]\right)^2\right] = \mathbb{E}\left[\left(\sum_{i=1}^n (X_i- \mathbb{E}[X_i])\right)^2\right]\\ &= \mathbb{E}\left[\sum_{i=1}^n\sum_{j=1}^n (X_i- \mathbb{E}[X_i])(X_j- \mathbb{E}[X_j])\right]\\ &= \sum_{i=1}^n\mathbb{E}\left[(X_i- \mathbb{E}[X_i])^2\right] + \sum_{i\neq j} \mathbb{E}\left[(X_i- \mathbb{E}[X_i])(X_j- \mathbb{E}[X_j])\right]\\ &= \sum_{i=1}^n\operatorname{Var}(X_i) + \sum_{i\neq j} 0 \\ &= \sum_{i=1}^n\operatorname{Var}(X_i) \end{align*}$$ as claimed. To go from the first to the second line, we expanded the square; to go from the second to the third, we divided the sum in two different sums, and then used linearity of expectation; and from the third to the fourth line, we used the definition of variance, and the assumption of pairwise uncorrelation.

Answer 2

Without loss of generality assume $E(X_i)=0$ (otherwise $Var(X)=Var(X-E(X))$). Assuming that $Var\left(\frac1n \sum_i X_i\right)=E\left(\frac1n \sum_i X_i\right)^2=\frac1{n^2}\left( \sum_i EX_i^2+\sum_{i,j:i\neq j}E(X_iX_j)\right)=\frac1{n^2}\left( \sum_i EX_i^2+\sum_{i,j:i\neq j}E(X_i)E(X_j)\right)\text{ (as they are uncorrelated) }=\frac1{n^2} \sum_i EX_i^2=\frac1{n^2} \sum_i Var(X_i)$

Answer 3

$E(\sum\limits_{i=1}^{n} X_i)^{2} =\sum\limits_{i,j=1}^{n} EX_iX_j=\sum\limits_{i=1}^{n} X_i^{2}+\sum_{i \neq j} EX_iX_j=\sum\limits_{i=1}^{n} EX_i^{2}+\sum_{i \neq j} EX_iEX_j$ because $X_i$ and $X_j$ are uncorrelated for $ i \neq j$. Hence $E(\sum\limits_{i=1}^{n} X_i)^{2}=\sum\limits_{i,j=1}^{n} var(X_i)+(EX_i)^{2})+\sum_{i \neq j} EX_iEX_j$ because $X_i$. Thus $E(\sum\limits_{i=1}^{n} X_i)^{2}=\sum\limits_{i,j=1}^{n} var(X_i)+E(\sum\limits_{i=1}^{n}X_i)^{2}$. Now take the second term on right side to the left side.