Finding the MLE estimate from a log function

Question

If I have the following function:
$$\ln L(\mu,\sigma^2)= -n \ln (\sqrt{2 \pi \sigma^2})+\sum_{i=1}^n \frac{-(x_i-\mu)^2}{2\sigma^2}$$

How can I find the MLE estimate $σ^2_\text{MLE}$?

I have already shown that $\mu = \bar{x}$

Answer 1

Take the derivative wrt $\sigma^2$:

$$-\frac n {\sqrt{2\pi\sigma^2}}\cdot \frac{2\pi}{2\sqrt{2\pi\sigma^2}}+\frac{\sum -(x-\mu)^2}{2}\cdot \frac{-1}{(\sigma^2)^2}=0\\ =\frac{-2n\pi}{4\pi\sigma^2}+\frac{\sum(x-\mu)^2}{2(\sigma^2)^2}\\ \frac{n}{2\sigma^2}=\frac{\sum(x-\mu)^2}{2(\sigma^2)^2}\\ \sigma^2=\frac{\sum(x-\mu)^2}{n}$$

Elaborate verification

To be rigorous, we need to show that this value of $\sigma^2$ maximizes the density. The first derivative is $$\frac{-n}{2\sigma^2}+\frac{\sum(x-\mu)^2}{(\sigma^2)^2}?0\\ -\sigma^2+\frac{\sum(x-\mu)^2}{n}?0$$

where the question mark represents an unknown relationship. We can see that the derivative is 0 at $\sum(x-\mu)^2/n$. When $\sigma^2$ is greater than this, the first derivative is negative, and when it is less than this, the first derivative is positive. So we can conclude that $\sigma^2$ is a global maximum. For fun, I also take the second derivative.

$$\frac{\partial^2}{(\partial \sigma^2)^2}\log l(\sigma^2|\mathbf x)=\frac{n}{2(\sigma^2)^2}-\frac{\sum(x-\mu)^2}{(\sigma^2)^3}\\ \sigma^2-\frac{2\sum(x-\mu)^2}{n}?0$$

This is zero when $\sigma^2$ is $\frac{2\sum(x-\mu)^2}{n}$, positive when it is greater than that, and negative when it is less than that. So there is an inflection point at $\frac{2\sum(x-\mu)^2}{n}$, the log likelihood is concave up when $\sigma^2$ is greater than that, and concave down when it is less than that.

Conclusion

$\sigma^2_{MLE}=\frac{\sum(x-\bar x)^2}{n}$

Answer 2

$$ \ln L(\mu,\sigma^2)= -n \ln \sqrt{2 \pi \sigma^2} - \frac 1 {2\sigma^2} \sum_{i=1}^n (x_i-\mu)^2 $$

You report that you have shown that the value of $\mu$ that maximizes this is $\overline x = (x_1+\cdots+x_n)/n.$

One could say that this is with $\sigma$ fixed, but more to the point, this result is the same regardless of $\sigma.$ And for any here who have not noticed this before, it is of interest to note that this is just the value of $\mu$ that minimizes $\sum_{i=1}^n (x_i-\mu)^2,$ and that the best way to see that is via the identity in algebra that says $$ \sum_{i=1}^n (x_i-\mu)^2 = n(\mu-\overline x)^2 + \sum_{i=1}^n (x_i-\overline x)^2. $$

What is sought is the pair $(\mu,\sigma^2)$ that maximizes $L(\mu,\sigma^2),$ and that is the same as the pair that maximizes $\ln L(\mu,\sigma^2),$ and that expression is used because it requires less work to get to the answer.

The maximizing value of $\mu$ with $\sigma^2$ fixed is the same regardless of $\sigma^2,$ and consequently we can just find the value of $\sigma^2$ that maximizes $L(\overline x,\sigma^2)$.

\begin{align} \ln L(\overline x,\sigma^2) & = -n \ln \sqrt{2 \pi \sigma^2} - \frac 1 {2\sigma^2} \sum_{i=1}^n (x_i-\overline x)^2 \\[8pt] & = -n\ln \sqrt{2\pi \sigma^2} - \frac{ns^2}{2\sigma^2} \\[8pt] & = \text{constant} -\frac n 2 \ln (\sigma^2) - \frac{ns^2}{2\sigma^2}. \tag 1 \end{align} I prefer to write that last line as $\text{constant}-n\ln\sigma - \dfrac{ns^2}{2\sigma^2}$ and differentiate with respect to $\sigma,$ but I've noticed various authorities preferring to treat $\sigma^2$ as a simple thing, so here I will do it that way.

The derivative of $(1)$ with respect to $\sigma^2$ is $$ - \frac n {2\sigma^2} + \frac{ns^2}{2(\sigma^2)^2} = n\cdot\frac{s^2-\sigma^2}{\sigma^4} \quad \begin{cases} >0 & \text{if } 0<\sigma^2<s^2 \\ =0 & \text{if } \sigma^2=s^2, \\ <0 & \text{if } \sigma^2 > s^2. \end{cases} $$ So this function of $\sigma^2$ increases on the interval $(0,\sigma^2]$ and decreases on the interval $[\sigma^2,+\infty).$ Therefore it has a glabal maximum at $\sigma^2 = s^2.$