MLE and log MLE

Question

I'm getting confused on likelihoods and the use of the log function.

negloglik = negative log likelihood

This statement on negloglik seems to be wrong to me:

" negloglik is an exponential scale, therefore small changes in negloglik represent a very large change in likelihood. "

Can anyone explain why it's true? The opposite seems to be the case if you ask me..

Answer 1

Let $L(x)$ denote the likelihood function and $N(x) = -\ln L(x)$ be the negative log of the likelihood function.

What happens to $L(x)$ if $N(x)$ is changed to $N(x) + \epsilon$? We get $L(x) = e^{-N(x)}$ so if $N'(x) = N(x) + \epsilon$ we get $$ L'(x) = e^{-N'(x)} = e^{-N(x)-\epsilon} = \frac{L(x)}{e^\epsilon}. $$

So for example

• if $\epsilon = 0.1$ then $L'(x) \approx 0.9 L(x)$,
• and if $\epsilon = 0.001$ then $L'(x) \approx 0.999 L(x)$,
• but if $\epsilon = 1$ then $L'(x) \approx 0.36 L(x)$.

Not sure if this is big enough or not...

UPDATE

Your specific example, if $L(x)$ goes from $0.1$ to $0.3$, then $N(x)$ goes from $2.3$ to $1.2$, more than 4 times the size of change in $L(x)$.

Answer 2

The general statement is perhaps inaccurate, as the effect is indeed dependent on the specific case. However, remember that the likelihood can (and it does in many cases) take values greater than one. Therefore it is not correct to say that the opposite is true.