Show that the maximum likelihood estimate of $\sigma^2$ is biased

334 Views Asked by At

Follow from this post How to derive the variance of this MLE estimator by deriving mle we get estimator for $\sigma^2$:

$$\hat \sigma^2 =\sum_{i=1}^N \frac{(y_i - \hat\beta\ x_i)^2}{n}\\$$

We already knew that $$E[\hat\beta]=\beta$$

$$Var[\hat\beta]= {\frac{\sigma^2}{\sum\limits_{i=1}^n x_{i}^2}} $$

I am trying to find $ E[\hat \sigma^2]$ by first try to expand $$(y_i - \hat\beta\ x_i)^2=y_i^2-2x_i\hat\beta y_i+\hat\beta^2x_i^2 $$ then take $$ E[\hat \sigma^2]=\sum_{i=1}^NE[y_i^2]+\sum_{i=1}^N2x_iE[\hat\beta y_i]+\sum_{i=1}^Nx_i^2E[\hat\beta^2]\,,$$

where $$E[y_i^2]=Var[y_i]+(E[y_i])^2=\sigma^2+\beta^2 x_i^2$$ and $$E[\hat\beta^2]=Var[\hat\beta]+(E[\hat\beta])^2={\frac{\sigma^2}{\sum\limits_{i=1}^n x_{i}^2}}+\beta^2$$

But I am not sure about $E[\hat\beta y_i]$ and when I add all terms up it seems really strange.