Find the MSE of the MLE

Question

Let $X_1,\ldots,X_n$ be i.i.d exponentially distributed r.v's with pdf:

$$ f_\theta(x) = \frac{1}{\theta}e^{-\frac{x}{\theta}}$$ such that $x\ge 0$.

I have found that the MLE is given by $$ \hat{\theta}(X)= \frac{1}{n}\sum_{i=1}^n X_i.$$

Note that $\hat{\theta}$ is unbiased as

$$ E(\hat{\theta}) = E\left(\frac{1}{n}\sum_{i=1}^n X_i\right) = \theta $$ since the r.v's are i.i.d.

But then I am having difficulties computing the MSE as $$ MSE = E_{\theta}\left( (\theta-\hat{\theta})^2 \right) $$ $$= E(\hat{\theta^2})+\theta^2-2\theta E(\hat{\theta})$$ $$ =E(\hat{\theta^2}) -\theta^2$$

So then $E(\hat{\theta^2}) = E\left(\frac{1}{n^2}\sum_{i=1}^n X_i^2\right) =E(X^2) = \int_0^\infty f_\theta dx = ... = \frac{2\theta^2}{n^2} $

$ \Rightarrow MSE = \frac{2\theta^2}{n^2} - \theta^2$

Supposedly the final answer should be $\frac{\theta^2}{n}$ but im not getting this at all...

Answer 1

Note that $$E[\hat{\theta}^2] \neq E\left[\frac{1}{n^2}\sum_{i=1}^n X_i^2\right].$$ An easy way to compute $E[(\theta-\hat{\theta})^2]$ is to observe that $E[(\theta-\hat{\theta})^2] = E[(\hat{\theta}-\theta)^2]=var(\hat{\theta})$. But $$var(\hat{\theta})=\frac{1}{n^2}\sum_{i=1}^{n}var(X_i)=\frac{n \theta^2}{n^2}=\frac{\theta^2}{n}.$$

Answer 2

It's not true that

$$ E(\hat{\theta^2}) = E\left(\frac{1}{n^2}\sum_{i=1}^n X_i^2\right).$$

Rather, one has

\begin{align} E(\hat{\theta^2}) &= E\left[\left(\frac{1}{n}\sum_{i=1}^n X_i\right)^2\right] \\ &= E\left[\frac{1}{n^2}\sum_{i,j} X_iX_j \right] \\ &= \frac{1}{n^2}E\left[\sum_{i=1}^n X_i^2 + \sum_{i\ne j} X_iX_j \right] \\ &= \frac{1}{n^2}E\left[\sum_{i=1}^n X_i^2 \right] + \frac{1}{n^2}E\left[\sum_{i\ne j} X_iX_j \right] \\ &= \frac{1}{n^2} \sum_{i=1}^nE\left[ X_i^2 \right] + \frac{1}{n^2} \sum_{i\ne j} E\left[X_iX_j \right] \\ &= \frac{1}{n^2} \sum_{i=1}^nE\left[ X^2 \right] + \frac{1}{n^2} \sum_{i\ne j} E[X_i]E[X_j] \\ &= \frac{1}{n^2} \sum_{i=1}^n 2\theta^2 + \frac{1}{n^2} \sum_{i\ne j} \theta^2 \\ &= \frac{1}{n^2} n2\theta^2 + \frac{1}{n^2} (n^2 - n)\theta^2 \\ &= \frac{\theta^2}{n^2} (n^2 - n + 2n) \\ &= \theta^2 + \frac{\theta^2}{n}. \end{align}