Let a point estimate for the sample variance be given as $\hat{\sigma}^2 = \frac{1}{n}\sum\limits_{i=1}^n(X_i- \bar{X})^2$ where $n$ is the number of samples. What is the bias in this estimate as a function of $n$ ? How much can you reduce the mean squared error of this estimate if you use the point estimate for the sample variance that gives the lowest mean squared error?
Do you have any idea? How can I solve this question?
Thanks in advance.
Recall that the estimator sample variance is normally defined as $$\hat\sigma^2 = \frac1{n-1}\sum_{i=1}^n (X_i - \bar X)^2.$$ This is because dividing by $n$ instead of $n-1$ gives us a biased estimator. Let $\mu$ and $\sigma^2$ be the population mean and variance. Then $$\begin{align*} \mathbb E\left[\tilde\sigma^2\right] &= \mathbb E\left[\frac1n\sum_{i=1}^n (X_i-\bar X)^2\right]\\ &= \mathbb E\left[\frac1n\sum_{i=1}^n (X_i-\mu)^2 - (\bar X - \mu)^2\right]\\ &= \mathbb E\left[\frac1n\sum_{i=1}^n (X_i-\mu)^2 \right] - \mathbb E\left[\frac1n\sum_{i=1}^n (\bar X-\mu)^2\right]\\ &=\frac1n\sum_{i=1}^n \operatorname{Var}(X_i) -\operatorname{Var}(\bar X)\\ &=\frac1n\sum_{i=1}^n \operatorname{Var}(X_i) - \operatorname{Var}\left(\frac1n\sum_{i=1}^n X_i\right)\\ &= \frac1n\sum_{i=1}^n \operatorname{Var}(X_i) - \frac1{n^2}\sum_{i=1}^n \operatorname{Var}(X_i)\\ &= \frac{n-1}n\sigma^2. \end{align*} $$ The bias is then $$ \begin{align*} \mathbb E\left[\tilde\sigma^2\right] - \sigma^2 &= \frac{n-1}n\sigma^2 -\sigma^2 = -\frac1n\sigma^2. \end{align*} $$ The variance of $\tilde\sigma^2$ is
$$ \operatorname{Var}\left(\tilde\sigma^2\right) = \frac{(n-1)^2}{n^3}\mu_4 - \frac{(n-1)(n-3)\mu_2^2}{n^3} $$ where $\mu_k$ is the $k^{\mathrm{th}}$ population central moment. (The derivation of this is quite tedious and I suggest you consult a reference such as this one.)
Therefore the MSE of $\tilde\sigma^2$ is
$$ \begin{align*} \operatorname{Var}\left(\tilde\sigma^2\right) + \operatorname{Bias}\left(\tilde\sigma^2,\sigma^2\right)^2 &= \frac{(n-1)^2}{n^3}\mu_4 - \frac{(n-1)(n-3)\mu_2^2}{n^3} + \frac1{n^2}(\sigma^2)^2. \end{align*} $$