Derivation of standard error of mean

Question

I was going through this wikipedia article on standard error. I could not understand the crucial step here. It goes like this:

This formula may be derived from what we know about the variance of a sum of independent random variables.

If $X_1, X_2 , \ldots, X_n$ are n independent observations from a population that has a mean $\mu$ and standard deviation $\sigma$ , then the variance of the total

$T = (X_1 + X_2 + \cdots + X_n)$ is $n\sigma^2$. Understood.

The variance of T/n must be $\frac{1}{n^2}n\sigma^2=\frac{\sigma^2}{n}$. Not understood.

And the standard deviation of T/n must be $\sigma/{\sqrt{n}}$ . Of course, T/n is the sample mean $\bar{x}$ .

I went to some basics:

$\displaystyle Var(X)=\frac{1}{n}\sum_{i=1}^{n}({x_i-\mu})^2$

$\displaystyle Var(X)=\frac{1}{n}\sum_{i=1}^{n}({x_i^2+\mu^2-2x_i\mu})$

$\displaystyle Var(X)=\frac{1}{n}\sum_{i=1}^{n}x_i^2+\mu^2-\frac{2}{n}\sum_{i=1}^{n}x_i\mu$

As Sample mean is an unbiased estimate of population mean, we get

$\displaystyle Var(X)=\frac{1}{n}\sum_{i=1}^{n}x_i^2-\mu^2$

$\displaystyle Var(X)=E(X^2)-(E(X))^2$

Nothing useful found from this.

Why is there a $1/n^2$ in that step to get variance ?

Answer 1

Let $Y$ be any random variable. Let $Z = Y/n$. Then $$Z^2 = \frac1{n^2} Y^2,$$ $$E(Z^2) = E\left(\frac1{n^2} Y^2\right) = \frac1{n^2} E(Y^2)$$ and therefore $$E\left(\left(\frac Yn\right)^2\right) = \frac1{n^2} E(Y^2).$$ Also, $$E(Z) = E\left(\frac1n Y\right) = \frac1n E(Y).$$ So from $Var(Y)=E(Y^2)-(E(Y))^2$ and $Var(Z)=E(Z^2)-(E(Z))^2,$ we find $$\begin{eqnarray} Var\left(\frac Yn\right) = Var(Z) &=& E(Z^2)-(E(Z))^2\\ &=& \frac1{n^2} E(Y^2) - \left(\frac1n E(Y)\right)^2 \\ &=& \frac1{n^2} \left(E(Y^2) - \left( E(Y)\right)^2 \right) \\ &=& \frac1{n^2} Var(Y). \end{eqnarray}$$ Now consider the case where $Y = T$.

Answer 2

Recall the definition of (population) variance: $$var\xi := E(\xi - E\xi)^{2}$$ for $\xi$ a random variable. Then we have $var\xi = E\xi^{2} - 2(E\xi)^{2} + (E\xi)^{2} = E\xi^{2} - (E\xi)^{2}$ so that $var(\xi/n) = E(\xi^{2})/n^{2} - (E\xi)^{2}/n^{2}.$

Thus we have

$$var(T/n) := var(X_{1}/n) + \cdots + var(X_{n}/n) = n\sigma^{2}/n^{2} = \sigma^{2}/n.$$

Answer 3

The only thing we need to prove here is that for any scalar constant $c$, and for a random variable $X$, $$\mathrm{Var}[cX] = c^2 \mathrm{Var}[X].$$ This follows from the property of expectation $$\mathrm{E}[cX] = c\mathrm{E}[X]$$ as follows: $$\begin{align*} \mathrm{Var}[cX] &= \mathrm{E}[(cX - \mathrm{E}[cX])^2] \\ &= \mathrm{E}[(cX - c\mathrm{E}[X])^2] \\ &= \mathrm{E}[c^2(X - \mathrm{E}[X])^2] \\ &= c^2 \mathrm{E}[(X - \mathrm{E}[X])^2] \\ &= c^2 \mathrm{Var}[X]. \end{align*}$$