I'm trying to derive the maximum likelihood estimator of $\sigma^2$ for multiple linear regression with $y \sim N(X\beta, \sigma^2I)$ and $\epsilon \sim N(0, \sigma^2)$. I'm stuck getting $\hat{\beta}$'s in place of my $\beta$'s.
I know I need to find $\frac{\partial}{\partial\sigma}$ from the log likelihood function, which I've boiled down to:
$$\frac{\partial}{\partial\sigma}\left[-n\ln\sigma-\frac{1}{2\sigma^2}\left(y-X\beta\right)^T\left(y-X\beta\right)\right] = -\frac{n}{\sigma}+\frac{1}{\sigma^3}\left(y-X\beta\right)^T\left(y-X\beta\right)$$
Solving for $\sigma^2$:
$$\sigma^2 = \frac{\left(y-X\beta\right)^T\left(y-X\beta\right)}{n} = \frac{\epsilon^T\epsilon}{n}$$.
From there, though, I don't see how to get to the final answer $\frac{\left(y-X\hat{\beta}\right)^T\left(y-X\hat{\beta}\right)}{n}$. Any help or nudge in the right direction is appreciated.
You can just plug in $\hat{\beta} = (X^TX)^{-1}X^T y$ in place of $\beta$. The idea is that you want to maximize the log-likelihood, $\ell(\beta,\sigma^2)$ with respect to both $\beta$ and $\sigma^2$, so you compute the partial derivatives and equate to zero, which gives you the following system of equations: $$ \frac{\partial \ell}{\partial \beta} \ell(\beta, \sigma^2)=0, \quad \frac{\partial \ell}{\partial \sigma^2} \ell(\beta, \sigma^2)=0 $$ This is just like any other system of equations, you can solve one and then plug the answer into the second one to get the final result.