Likelihood / maximum likelihood of this distribution?

78 Views Asked by At

Suppose we draw $n$ independent random reals $x$ in range $0 \le x < \infty $ from the exponential distribution with $P(x) = \mu e^{-\mu x}$.

  1. What's the liklihood of drawing a particular set of $x_i,\ldots,x_n$ for a given value of $\mu$?
  2. What is the maximum liklihood estimate of $\mu$ given your answer to (1)?

My work

  1. $$L = \mu e^{-\mu x_1} \times \mu e^{-\mu x_2} \times\cdots\times \mu e^{-\mu x_n} = \mu^n e^{-\mu \sum x_i} $$

  2. We can maximize the liklihood by maximizing the log liklihood. So, then $$\log L = n\log \mu - \mu \sum x_i$$ and then $$\frac{\partial \log L}{\partial\mu} = \frac n \mu - \sum x_i = 0 \implies \frac n \mu = \sum x_i \implies \mu = \frac n {\sum x_i}$$

1

There are 1 best solutions below

0
On

The only deficiency I see you your argument is the tacit assumption that every critical point is a global maximum point.

$$ \frac {\partial\log L(\mu)}{\partial\mu} = \frac n \mu - \sum_i x_i \quad \begin{cases} >0 & \text{if } 0\le\mu< n/\sum_ix_i, \\[6pt] =0 & \text{if } \mu = n/\sum_ix_i, \\[6pt] <0 & \text{if } \mu > n/\sum_ix_i. \end{cases} $$ That proves there is a global maximum at the critical point.

That is not the only way to do it. One could also say:

  • $L(\mu)=0$ when $\mu=0$ and $L(\mu)\to0$ as $\mu\to+\infty$ and $L(\mu)>0$ when $\mu$ is between those extremes, and $L$ is a continuous function; therefore there must exist a global maximum point; and

  • since $L$ is everywhere differentiable, the global maximum must occur at a point where $L'=0;$ and

  • there is only one point where $L'=0.$

Just setting the derivative to $0$ and concluding the solution is the bottom-line answer gets you into trouble in some cases. One of those is estimating the minimum of a Pareto distribution, because in that case there is an endpoint maximum. (Look for "endpoint" in the index of a calculus textbook. You'll probably find something about that.)