Show that $ E\Big[\sum_{i=1}^{n}(X_i - \bar{X_n})^2\Big] = (n-1)\sigma^2 $

Question

$X_1, X_2, ..., X_n$ are i.i.d random variables with mean $\mu$ and variance $\sigma^2$. $\bar{X_n} = S_n/n$ where $S_n = X_1 + X_2 + ... + X_n$. Show that

$$ E\Big[\sum_{i=1}^{n}(X_i - \bar{X_n})^2\Big] = (n-1)\sigma^2 $$

Attempt:

$$ E\Big[\sum_{i=1}^{n}(X_i - X_n)^2\Big] = E\Big[\sum_{i=1}^{n}~X_i^2 - 2X_i\bar{X_n} + \bar{X_n}^2\Big] \tag{1} $$

$$ = \sum_{i=1}^{n}E\Big(X_i^2 - 2X_i\bar{X_n} + \bar{X_n}^2) \tag{2} $$

$$ = \sum_{i=1}^{n}\Big(E(X_i^2) - 2E(X_i\bar{X_n}) + E(\bar{X_n}^2)\Big) \tag{3} $$

I don't know how to proceed.

Answer 1

Observe that $$ (X_i-\mu)^2=(X_i-\bar{X})^2+(\bar{X}-\mu)^2+2(X_i-\bar{X})(\bar{X}-\mu) $$ and moreover $$ \sum_{i=1}^n (X_i-\bar{X})(\bar{X}-\mu)=(\bar{X}-\mu) \sum_{i=1}^n (X_i-\bar{X})=0 $$whence $$ \sum_{i=1}^n(X_i-\mu)^2=\sum_{i=1}^n(X_i-\bar{X})^2+\sum_{i=1}^n(\bar{X}-\mu)^2. $$ In particular $$ \begin{align} n\sigma^2=n\text{Var}(X_1)=E\left[\sum_{i=1}^n(X_i-\mu)^2\right] &= E\left[\sum_{i=1}^n(X_i-\bar{X})^2\right]+ E\left[\sum_{i=1}^n(\bar{X}-\mu)^2\right]\\ &=E\left[\sum_{i=1}^n(X_i-\bar{X})^2\right]+n\text{Var}(\bar{X})\\ &=E\left[\sum_{i=1}^n(X_i-\bar{X})^2\right]+\sigma^2 \end{align} $$ from which the desired result follows.

Answer 2

$$E\Big[\sum_{i=1}^{n}(X_i - X_n)^2\Big]=\sum_{i=1}^{n}\Big(E(X_i^2) - 2E(X_i\bar{X_n}) + E(\bar{X_n}^2)\Big)$$

We have

$$E(X_i^2)=\sigma^2+\mu^2$$

$$E(X_i\bar{X_n})=\frac{1}{n}\sum_{k=1}^{n}{E(X_i X_k)}=E(X_i^2)+\frac{1}{n}\sum_{k=1, k \ne i}^{n}{E(X_i)E( X_k)}$$ because $X_i$'s are independent. Thus, $$E(X_i\bar{X_n})=\sigma^2+\mu^2+\frac{n-1}{n}\mu^2$$

Also,

$$E(\bar{X_n}^2)=\frac{1}{n^2}\sum_{i=1}^{n}\sum_{j=1}^{n}{E(X_iX_j)}=\frac{1}{n^2}\sum_{i=1}^{n}{E(X_i^2)}+{\frac{1}{n^2}\sum_{i=1}^{n}{\sum_{j=1, i\ne j}^{n}{E(X_i)E(X_j)}}}$$

Therefore, $$E(\bar{X_n}^2)=\frac{n}{n^2}(\sigma^2+\mu^2)+\frac{n(n-1)}{n^2}\mu^2$$

Finally add them up, and you have the results wanted.