Fisher information for exponential distribution

31.3k Views Asked by At

Let $y_1, \dots,y_n$ be i.i.d. random variables from $Exp(\theta)$, where $\theta$ is scale parameter. I have to find Fisher information $i(\theta)$.

The density function is $$f(y)=\frac{1}{\theta}e^{-\frac{y}{\theta}}$$

and the likelihood function $$L(\theta)=\frac{1}{\theta^n}e^{-\frac{\sum^{n}_{i=1}y_i}{\theta}}$$

The log-likelihood is

$$l(\theta)=-n\ln\theta-\frac{\sum^{n}_{i=1}y_i}{\theta}$$

Now, the score function $$l_*(\theta)=\frac{dl(\theta)}{d\theta}=-\frac{n}{\theta}+\frac{1}{\theta^2}\sum^{n}_{i=1}y_i$$

given the MLE

$$\hat \theta=\frac{\sum^{n}_{i=1}y_i}{n}$$

I differentiate again to find the observed information

$$j(\theta)=-\frac{dl_*(\theta)}{d\theta}=-(\frac{n}{\theta^2}-\frac{2}{\theta^3}\sum^{n}_{i=1}y_i)$$

and Finally fhe Fisher information is the expected value of the observed information, so

$$i(\theta)=\mathbb{E}(j(\theta))=-\frac{n}{\theta^2}+\frac{2}{\theta^3}n\theta=\frac{n}{\theta^2}$$

Is everything correct?

1

There are 1 best solutions below

1
On BEST ANSWER

Yes it's correct. Very well done.

This doesn't simplify the work a lot in this case, but here's an interesting result . . . In the case of $n$ i.i.d. random variables $y_1,\dots,y_n$ , you can obtain the Fisher information $i_{\vec y}(\theta)$ for $\vec y$ via $n \cdot i_y (\theta$) where $y$ is a single observation from your distribution.

Here $\ell(\theta) = \ln( \frac{1}{\theta} e^{-y/\theta}) = -y/\theta - \ln(\theta) \implies \frac{\partial}{\partial \theta} \ell (\theta) = \frac{y}{\theta^2} - \frac{1}{\theta} \implies \frac{\partial^2}{\partial \theta^2} \ell(\theta) = - \frac{2y}{\theta^3} + \frac{1}{\theta^2}$ \begin{align*} i_y(\theta) &= - E \left[ \frac{\partial^2}{\partial \theta^2} \ell(\theta) \right] = -E \left[ - \frac{2y}{\theta^3} + \frac{1}{\theta^2} \right] = \dfrac{2 \theta}{\theta^3} - \dfrac{1}{\theta^2} = \dfrac{1}{\theta^2} \end{align*} and multiplying by $n$ gives Fisher information $n/\theta^2$.