Error when trying to derive variance of sample mean

35 Views Asked by At

Assume that $X$ is a random variable with mean $E[X]$ variance $\sigma^2$. Let $\mu = \frac{1}{N}\sum_{n=1}^N X_n$, where $X_n$ is i.i.d with respect to $n$ having mean $E[X]$ and variance $\sigma^2$, be an estimator of $E[X]$. I tried to show that $Variance(E[X] - \mu) = \frac{\sigma^2}{N}$ but I have made some error which I'm blind to... An approach which yields the correct answer can be found at http://sepwww.stanford.edu/sep/prof/pvi/rand/paper_html/node16.html.

I'm proceeding as follows: Let $e = E[X]-\mu$ then \begin{align} Variance(e) &= E[(e -E[e])^2] \\ &= E[e^2] - E[e]^2 \\ &= E[e^2] & \text{unbiased estimator} \\ &= E[(E[X]-\mu)^2] \\ &= E[E[X]^2 - E[X]\mu + \mu^2] \\ &= E[\mu^2] - E[X]^2 & \text{since }E[\mu] = E[X] \\ &= E\left[\left(\frac{1}{N}\sum_{n=1}^N X_n\right)^2\right] - E[X]^2 \\ &= E\left[\frac{1}{N^2}\sum_{m=1}^N\sum_{n=1}^N X_m X_n\right] - E[X]^2 \\ &= \left(\frac{1}{N^2}\sum_{m=1}^N\sum_{n=1}^N E[X_m X_n]\right) - E[X]^2 \\ &= \left(\frac{1}{N^2}\sum_{n=1}^N E[X_n^2]\right) - E[X]^2 & \text{due to independence} \\ &= \left(\frac{1}{N^2}\sum_{n=1}^N E[X^2]\right) - E[X]^2 & \text{from assumption on distr. of $X_n$} \\ &= \frac{1}{N} E[X^2] - E[X]^2 \\ &= \frac{1}{N} E[X^2] - E[X]^2 \\ &= \frac{1}{N} (\sigma^2 + E[X]^2) - E[X]^2 \\ &= \frac{\sigma^2}{N} + (\frac{1}{N} - 1) E[X]^2 \\ \end{align}

EDIT: Correcting the error pointed out by NCh gives the correct answer. Continuing from the 9th row \begin{align} &= \left(\frac{1}{N^2}\sum_{m=1}^N\sum_{n=1}^N E[X_m X_n]\right) - E[X]^2 \\ &= \frac{1}{N^2}\left(\left(\sum_{n=1}^N E[X_n^2]\right) + \left(\sum_{m=1}^N\sum_{n\in\{1,\dots,N\}-\{m\}} E[X_m]E[X_n]\right)\right) - E[X]^2 & \text{due to independence} \\ &= \frac{1}{N^2}\left(\left(\sum_{n=1}^N E[X^2]\right) + \left(\sum_{m=1}^N\sum_{n\in\{1,\dots,N\}-\{m\}} E[X]^2\right)\right) - E[X]^2 & \text{from assumption on distr. of $X_n$} \\ &= \frac{1}{N^2}(NE[X^2] + N(N-1)E[X]^2) - E[X] \\ &= \frac{1}{N}(E[X^2] - E[X]^2) \\ &= \frac{\sigma^2}{N} \\ \end{align}

1

There are 1 best solutions below

0
On BEST ANSWER

An error is when you went from 9th to 10th raw: $$\mathbb E[X_m X_n]\neq \mathbb E[X_n^2]$$ for $m\neq n$.

By independence, $$\mathbb E[X_m X_n]=\mathbb E[X_m]\mathbb E[X_n]=\mathbb E[X]^2$$

Note also that by properties of variance, $$ Var(E[X]-\mu) = Var(\mu) = Var\left(\frac{\sum_{i=1}^N X_i}{N}\right) = \frac{1}{N^2} Var\left(\sum_{i=1}^N X_i\right) = \frac{1}{N^2} \left(\sum_{i=1}^N Var(X_i)\right) = \frac{N \sigma^2}{N^2} = \frac{\sigma^2}{N} $$