Maximum Likelihood Estimator for Logarithmic Distribution

695 Views Asked by At

I am trying to calculate the MLE for the logarithmic distribution. It holds

$$ P(X=k) = -\frac{p^k}{k \cdot \ln(1-p} $$

Thus, the ML function is

$$L_p(X_1,\dots, X_n) = \prod_{i=1}^{n} -\frac{p^{x_i}}{x_i \cdot \ln(1-p)} = p^{\sum_{i=1}^n x_i} \left(\frac{1}{\ln(1-p)}\right)^n \prod_{i=1}^n -\frac{1}{x_i} $$

and the log likelihood function is

\begin{align} \log L_p(X_1,\dots,X_n)& = \sum_{i=1}^n x_i \cdot \log(p) + n \cdot \log\left(\frac{1}{\log(1-p)}\right) + \log\left(\prod_{i=1}^n -\frac{1}{x_i}\right) \\&= \sum_{i=1}^n x_i \cdot \log(p) + n \cdot -\log(\log(1-p)) + \log\left(\prod_{i=1}^n -\frac{1}{x_i}\right) \end{align}

So:

$$\frac{\partial L_p}{\partial p} = \frac{1}{p} \sum_{i=1}^n x_i + n \frac{1}{\log(1-p)(1-p)} \overset{!}{=} 0$$

This is equivalent to:

$$\frac{1}{n} \sum_{i=1}^n x_i = - \frac{p}{\log(1-p)(1-p)}$$

Now I don't know how to go on. How do I get the estimator for $p$?

Thanks in advance, for helping !

2

There are 2 best solutions below

0
On

Technically, I suspect that you have to write the log likelihood function as

$$ \sum_{i=1}^n x_i \cdot \log(p) + n \cdot \log\left(\frac{-1}{\log(1-p)}\right) + \log\left(\prod_{i=1}^n \frac{1}{x_i}\right) $$

since $log(1-p)$ is negative, so $log(\frac{1}{log(1-p)})$ is undefined. I got the same final equation as you for the MLE, though.

I don't see a way to get a closed form solution, so I think you need to use numerical methods/root-finding algorithms to find the root(s) of $$\frac{1}{\hat{p}} \sum_{i=1}^n x_i + n \frac{1}{\log(1-\hat{p})(1-\hat{p})} = 0$$

0
On

I precise that I am not familiar with this domains you tagged.

Let me suppose that you look for $p$ such that $$\frac{1}{n} \sum_{i=1}^n x_i = - \frac{p}{\log(1-p)(1-p)}$$ where the value of the lhs is known.

This is the same as solving for $p$ the equation $$-\frac 1p \log(1-p)(1-p) =a \qquad \text{where} \qquad a=\frac n{\sum_{i=1}^n x_i}$$ and the lhs is a pretty nice function (plot it).

A very simplistic approximation of the lhs is $$-\frac 1p \log(1-p)(1-p) \sim 1-\frac 12(p^{1.3}+p^{2.6})$$ which would give as an estimate $$p_0=\Bigg[\frac{\sqrt{9-8 a}-1}{2}\Bigg]^{\frac {10}{13}} $$ The solution will be easily obtained using very few iterations of Newton method $$p_{n+1}=\frac{p_n ((a+1) p_n+(2-p_n) \log (1-p_n))}{p_n+\log (1-p_n)}$$

Just to show a few numbers

$$\left( \begin{array}{ccccc} a & p_0 & p_1 & p_2 & \text{exact} \\ 0.05 & 0.973965 & 0.990822 & 0.989085 & 0.989044 \\ 0.10 & 0.947100 & 0.975995 & 0.973133 & 0.973082 \\ 0.15 & 0.919331 & 0.956826 & 0.953390 & 0.953343 \\ 0.20 & 0.890577 & 0.933684 & 0.930116 & 0.930080 \\ 0.25 & 0.860741 & 0.906737 & 0.903375 & 0.903350 \\ 0.30 & 0.829712 & 0.876061 & 0.873142 & 0.873127 \\ 0.35 & 0.797359 & 0.841674 & 0.839341 & 0.839333 \\ 0.40 & 0.763526 & 0.803555 & 0.801865 & 0.801861 \\ 0.45 & 0.728024 & 0.761651 & 0.760580 & 0.760578 \\ 0.50 & 0.690622 & 0.715880 & 0.715332 & 0.715332 \\ 0.55 & 0.651034 & 0.666131 & 0.665952 & 0.665952 \\ 0.60 & 0.608897 & 0.612262 & 0.612254 & 0.612254 \\ 0.65 & 0.563738 & 0.554098 & 0.554036 & 0.554036 \\ 0.70 & 0.514924 & 0.491419 & 0.491084 & 0.491084 \\ 0.75 & 0.461572 & 0.423957 & 0.423166 & 0.423166 \\ 0.80 & 0.402390 & 0.351379 & 0.350039 & 0.350039 \\ 0.85 & 0.335341 & 0.273265 & 0.271443 & 0.271441 \\ 0.90 & 0.256830 & 0.189082 & 0.187101 & 0.187099 \\ 0.95 & 0.159033 & 0.098164 & 0.096722 & 0.096722 \end{array} \right)$$

Edit

A better $p_0$ can be generated approximating $$f(p)=-(1-p)\log(1-p)\qquad \text{by} \qquad g(p)=p(1-p)(\alpha+\beta p+\gamma p^2)$$ Minimizing, with respect to $(\alpha,\beta,\gamma)$ the norm $$\Phi(\alpha,\beta,\gamma)=\int_0^1 \big[f(p)-g(p)\big]^2\,dp$$ leads to $$\alpha=-\frac{49}{20}\qquad \qquad \beta=\frac{203}{60}\qquad \qquad \gamma=-\frac{259}{120}$$ which makes that we need now to solve for $p$ the equation $$p(1-p)(\alpha+\beta p+\gamma p^2)= ap$$ which is a cubic with one real root given by $$p_0=\frac{1}{63} \left(29-\sqrt{1298} \sinh \left(\frac{1}{3} \sinh ^{-1}\left(\frac{1}{649} \sqrt{\frac{2}{649}} (102060 a-72403)\right)\right)\right)$$ The iterates will be $$p_{n+1}=\frac{p_n+\log (1-p_n)}{\log (1-p_n)+1-a}$$ Repeating the same table as before $$\left( \begin{array}{ccccc} a & p_0 & p_1 & p_2 & \text{exact} \\ 0.05 & 0.981261 & 0.989673 & 0.989050 & 0.989044 \\ 0.10 & 0.961424 & 0.973919 & 0.973087 & 0.973082 \\ 0.15 & 0.940327 & 0.954123 & 0.953346 & 0.953343 \\ 0.20 & 0.917764 & 0.930652 & 0.930081 & 0.930080\\ 0.25 & 0.893478 & 0.903668 & 0.903351 & 0.903350 \\ 0.30 & 0.867132 & 0.873231 & 0.873127 & 0.873127 \\ 0.35 & 0.838282 & 0.839336 & 0.839333 & 0.839333 \\ 0.40 & 0.806326 & 0.801910 & 0.801861 & 0.801861 \\ 0.45 & 0.770425 & 0.760804 & 0.760578 & 0.760578 \\ 0.50 & 0.729386 & 0.715776 & 0.715332 & 0.715332 \\ 0.55 & 0.681491 & 0.666490 & 0.665953 & 0.665952 \\ 0.60 & 0.624389 & 0.612589 & 0.612254 & 0.612254 \\ 0.65 & 0.555658 & 0.554043 & 0.554036 & 0.554036 \\ 0.70 & 0.475971 & 0.491719 & 0.491085 & 0.491084 \\ 0.75 & 0.393972 & 0.426015 & 0.423189 & 0.423166 \\ 0.80 & 0.320678 & 0.353488 & 0.350077 & 0.350039 \\ 0.85 & 0.259426 & 0.272093 & 0.271443 & 0.271441 \\ 0.90 & 0.208371 & 0.189219 & 0.187125 & 0.187099 \\ 0.95 & 0.164984 & 0.117574 & 0.099978 & 0.096722 \end{array} \right)$$