Calculating the expected value from this kind of variance

Question

Suppose the variance of an iid sequence of random variables is formed according to

$$\widehat \sigma = \frac{1}{n} \sum^{n}_{m=1}(X_m - \widehat \mu )^2$$

$\widehat \mu$ is the sample mean. Find the expected value of this estimator and show that it is biased

I don't know how to calculate the expected value from this kind of variance,can anyone teach me?

Answer 1

We have to show that $\mathbb E(s^2)\neq \sigma^2$. I write down the full calculation with some explanations. If there comes a step where you think you can go on by yourself just don´t continue reading.

$\mathbb E(s^2)=\mathbb E\left[\frac{1}{n}\sum_{i=1}^n (X_i-\overline X )^2\right]$

$=\frac{1}{n}\mathbb E\left[\sum_{i=1}^n (X_i-\overline X)^2 \right] \quad | \pm \mu$

$=\frac{1}{n}\mathbb E\left[\sum_{i=1}^n \left[(X_i-\mu)-(\overline X-\mu) \right]^2 \right] \quad$

multipliying out

$=\frac{1}{n}\mathbb E\left[\sum_{i=1}^n \left[(X_i-\mu)^2-2(\overline X-\mu)(X_i-\mu)+(\overline X-\mu)^2 \right]\right] \quad$

writing for each summand a sigma sign

$=\frac{1}{n}\mathbb E\left[\sum_{i=1}^n (X_i-\mu)^2-2(\overline X-\mu)\sum_{i=1}^n(X_i-\mu)+\sum_{i=1}^n(\overline X-\mu)^2 \right] \quad$

$=\frac{1}{n}\mathbb E\left[\sum_{i=1}^n (X_i-\mu)^2-2(\overline X-\mu)\color{red}{\sum_{i=1}^n(X_i-\mu)}+n(\overline X-\mu)^2 \right] \quad$

---

transforming the red term

$\sum_{i=1}^n(X_i-\mu)=n\cdot \overline X-n\cdot \mu$

Thus $2(\overline X-\mu)\color{red}{\sum_{i=1}^n(X_i-\mu)}=2(\overline X-\mu)\cdot (n\cdot \overline X-n\cdot \mu)=2n( \overline X- \mu)^2$

---

$=\frac{1}{n}\mathbb E\left[\sum_{i=1}^n (X_i-\mu)^2-2n( \overline X- \mu)^2+n(\overline X-\mu)^2 \right] \quad$

$=\frac{1}{n}\mathbb E\left[\sum_{i=1}^n (X_i-\mu)^2-n( \overline X- \mu)^2\right] \quad$

$=\frac{1}{n}\left[\sum_{i=1}^n \mathbb E\left[(X_i-\mu)^2\right]-nE\left[( \overline X- \mu)^2\right]\right] \quad$

We know, that $\mathbb E\left[(X_i-\mu)^2\right]=\sigma^2$ and $E\left[( \overline X- \mu)^2\right]=\sigma_{\overline x}^2=\frac{\sigma^2}{n}$ Thus we get

$=\frac{1}{n}\left[n \cdot \sigma ^2-n \frac{\sigma ^2}{n}\right]=\frac{1}{n} \sigma^2 \cdot (n-1)$

Answer 2

Showing simply that it is biased is easier than finding its expected value: $$ (X_m - \mu )^2 = \big( (X_m - \widehat\mu) + (\widehat\mu - \mu) \big)^2 = \underbrace{ (X_m-\widehat\mu)^2}_A + \underbrace{2(X_m-\widehat\mu)(\widehat\mu-\mu)}_B + \underbrace{(\widehat\mu-\mu)^2}_C $$ The expected value of the sum of the terms labeled $B$ is $0,$ because the sum of $X_m-\widehat\mu$ is $0$ (later note: This does not mean that $X_m-\widehat\mu)$ is $0;$ but only that their sum is $0$), and the other factor, $2(\widehat\mu-\mu),$ does not change as $m$ goes from $1$ to $n.$

Therefore the sum of the terms labeled $A,B,C$ is the sum of those labeled $A$ and those labeled $C.$

Therefore the expected value of the sum of the terms on the left is bigger than the expected value of the sum of the terms labeled $A.$

And the expected value of the sum of the terms on the left is $n\sigma^2.$