How to derive the likelihood and loglikelihood of the poisson distribution

Question

As the title suggests, I'm really struggling to derive the likelihood function of the poisson distribution (mostly down to the fact I'm having a hard time understanding the concept of likelihood at all).

I've watched a couple videos and understand that the likelihood function is the big product of the PMF or PDF of the distribution but can't get much further than that. The question is as follows:

"Random variables $X_1, \dots, X_n$ are independent and identically distributed (IID) from a $Poisson(θ)$ distribution. Suppose a random sample $x = (x_1, \dots, x_n)$ has been observed.

(a) Write down the likelihood function $L(θ)$ based on the observed sample."

If anyone could help show me the process for deriving the likelihood function I would really appreciate it. Thanks!

Answer 1

For a Poisson random variable $X$, the probability mass function (PMF) is given by: $$P(X=x|\theta)=f(x)=e^{-\theta} \frac{\theta^x}{x!},\ \ x\in \{0,1,\ldots,\infty\},\theta>0$$

As for the likelihood function, considering an i.i.d. (independent and identically distributed) sample $x_1, x_2,\ldots,x_n$, from a Poisson variable,

$$L(\theta|x_1,x_2,\ldots,x_n)=P(X=x_1|\theta)P(X=x_2|\theta)\cdots P(X=x_n|\theta)$$

$$L(\theta|x_1,x_2,\ldots,x_n)=e^{-\theta} \frac{\theta^{x_1}}{x_1!}\cdots e^{-\theta} \frac{\theta^{x_n}}{x_n!}=e^{-n\theta}\frac{\theta^{x_1+x_2+\ldots+x_n}}{x_1!x_2!\cdots x_n!}$$ $$L(\theta|x_1,x_2,\ldots,x_n)=e^{-n\theta}\frac{\theta^{\sum_{i=1}^n x_i}}{\prod_{i=1}^n x_i!}$$

Now, for the log-likelihood: just apply natural log to last expression.

$$\ln L(\theta|x_1,x_2,\ldots,x_n)=-n\theta + \left(\sum_{i=1}^n x_i\right)\ln \theta - \ln(\prod_{i=1}^n x_i!).$$

If your problem is finding the maximum likelihood estimator $\hat \theta$, just differentiate this expression with respect to $\theta$ and equate it to zero, solving for $\hat \theta$. As $\theta$ is not present in the last term you can easily find that $$\hat \theta=\frac{\sum_{i=1}^n x_i}{n}.$$

Answer 2

The pdf (or pmf) of $\mathsf{Pois}(\lambda)$ is $f(x|\lambda) = e^{-\lambda}\lambda^x/x!,$ for $\lambda > 0$ and $x = 0, 1,2, \dots .$

The likelihood of a sample $\mathbf{x} = (x_1, x_2, \dots, x_n)$ is defined as the product $$f(\mathbf{x}|\lambda) = \prod_{i=1}^n f(x|\lambda) = \prod_{i=1}^n \frac{e^{-\lambda}\lambda^{x_i}}{x_i!} = \frac{e^{-n\lambda}\lambda^{\sum_i x_i}}{\prod_i x_i!} = \frac{e^{-n\lambda}\lambda^t}{\prod_i x_i!} \propto e^{-n\lambda}\lambda^t,$$ where $ t = \sum_{i=1}^n x_i$ is the total of the $n$ observations.

Also, in this setting $f(\mathbf{x}|\lambda)$ is viewed as a function of the variable $\lambda$ for fixed observed values of the $x_i$ and $t.$ It is customary to specify a likelihood function 'up to a constant factor', so in the last step of the displayed equation we have dropped the constant in the denominator. [The symbol $\propto$ is read "proportional to".]

The log-likelihood is the logarithm (usually the natural logarithm) of the likelihood function, here it is $$\ell(\lambda) = \ln f(\mathbf{x}|\lambda) = -n\lambda +t\ln\lambda.$$

One use of likelihood functions is to find maximum likelihood estimators. Here we find the value of $\lambda$ (expressed in terms of the data) that maximizes the likelihood function $f(\mathbf{x}|\lambda).$

Maximizing $\ell(\lambda)$ accomplishes the same goal. For Poisson data we maximize the likelihood by setting the derivative (with respect to $\lambda)$ of $\ell(\theta)$ equal to $0$, solving for $\lambda$ and verifying that the result is an absolute maximum.

The derivative of the log-likelihood is $\ell^\prime(\lambda) = -n + t/\lambda.$ Setting $\ell^\prime(\lambda) = 0$ we obtain the equation $n = t/\lambda.$ Solving this equation for $\lambda$ we get the maximum likelihood estimator $\hat \lambda = t/n = \frac{1}{n}\sum_i x_i = \bar x.$ I will leave it to you to verify that $\bar x$ is truly the maximum.

Example: Suppose we get the $n = 5$ values $12, 5, 12, 9, 8,$ which sum to 46 and which have a sample mean $\bar x = 9.2.$ Then the likelihood function is $f(\mathbf{x}|\lambda) = e^{-5\lambda}\lambda^{46}.$ The graph below illustrates the maximum of the likelihood curve does indeed occur at $\hat \lambda = 9.2.$ [The five observations were simulated from $\mathsf{Pois}(\lambda=10),$ so $\hat \lambda = 9.2$ is not a bad estimate of $\lambda$ using only $n = 5$ observations.]