Maximum Likelihood estimator for negative binomial

Question

Let $X$ be negatively binomially distributed to parameters $n \in \mathbb{N}$ and $p \in] 0,1[$, i.e. $$ \mathbb{P}(X=k)=\binom{n+k-1}{k} p^n(1-p)^k, \quad k \in \mathbb{N}_0. $$ Let $n \in \mathbb{N}$ be known and $p \in\, ]0,1[$ be unknown. Show that the maximum likelihood estimator for $p$ is given by. $$ \widehat{p}(k)=\frac{n}{k+n}. $$

Answer 1

You have to be clear about what constitutes a sample, before you calculate the MLE.

If your sample is a collection of independent and identically distributed observations that each follow the aforementioned negative binomial distribution, i.e. $$X_1, X_2, \ldots, X_m \sim \operatorname{NegativeBinomial}(n, p), \\ \Pr[X_i = k] = \binom{n+k-1}{k} p^n (1-p)^k, \quad k \in \{0, 1, 2, \ldots \}, \quad i \in \{1, 2, \ldots, m\},$$ then your likelihood function for $p$ is $$\mathcal L(p \mid n, x_1, \ldots, x_m) \propto \prod_{i=1}^m \binom{n+x_i-1}{x_i} p^n (1-p)^{x_i}.$$

Notice here that the sample size is $m$, not $n$; and the product is indexed over $i$ from $1$ to $m$. But this is not at all what you did. You somehow took the sample to be $n$, and indexed the product over $k$, which makes no sense.

Now, if your sample is a single observation $X$ from a negative binomial distribution, that is to say, $m = 1$, then $x_1 = k$ and you can immediately see how the product notation disappears. The likelihood is simply written

$$\mathcal L(p \mid n, k) \propto \binom{n+k-1}{k} p^n (1-p)^k.$$ Since $k$ and $n$ are not functions of $p$, we can ignore the constant of proportionality $\binom{n+k-1}{k}$ with respect to $p$ and write $$\mathcal L(p \mid n,k) \propto p^n (1-p)^k.$$ The log-likelihood is then $$\ell(p \mid n,k) \propto n \log p + k \log(1-p)$$ and the critical points of the log-likelihood satisfy $$0 = \frac{\partial \ell}{\partial p} = \frac{n}{p} - \frac{k}{1-p}.$$ Solving for $p$ yields the unique critical point $$\hat p = \frac{n}{k+n}$$ which is the claimed MLE. In this case, the realized statistic is the single observed value $k$.

As an exercise, what is the MLE in the general case with sample size $m$, the likelihood of which I provided above? Note that it will be a function of $m$, $n$, and a sufficient statistic of the sample $(x_1, \ldots, x_m)$.

Answer 2

Note that $$ \ln g(k)=\ln \binom{n+k-1}{k}+n \ln p+k \ln (1-p) $$ for $k \in \mathbb{N}$. Thus, the log-likelihood function for $\boldsymbol{k}=\left(k_1, k_2, \ldots, k_r\right) \in \mathbb{N}^r$ is. $$ \ln L_k(p)=rn \ln p+y \ln (1-p)+C, \quad p \in(0,1) $$ where $y=\sum_{i=1}^r k_i$ and $C=\sum_{i=1}^r \ln \binom{n+k_i-1}{k_i}$. From this follows $$ \frac{\partial}{\partial p} \ln L_k(p)=\frac{rn}{p}-\frac{y}{1-p} $$ The derivative is 0 if $$ p=\frac{rn}{rn+y}=\frac{n}{n+m}, $$ where, as usual, $m=\frac{y}{r}$. Finally $$ \frac{\partial^2}{\partial p^2} \ln L_{\boldsymbol{k}}(p)=-r n / p^2-y /(1-p)^2<0, $$ so the maximum occurs at the critical point.