Asymptotic Variance of MLE Exponential

Question

Suppose we have a random sample $(X_{1},.....,X_{n})$, where $X_{i}$ follows an Exponential Distribution with parameter $\lambda$, hence:

$F(x) = 1 - exp(-\lambda x)$

$\mathbb{E}(X_{i}) = \dfrac{1}{\lambda}$

$ Var(X_{i}) =\dfrac{1}{\lambda^{2}}$

I know that the MLE estimator $\hat{\lambda} = \dfrac{n}{\sum_{i=1}^{n}X_{i}}$, asymptotically follows a normal distribution, but I'm interested in his variance. So, since $\sqrt n (\hat{\lambda} - \lambda)\stackrel{D}{\rightarrow} \mathcal{N}(0, \sigma^{2}) $

I was thinking about dealing with:

\begin{align} \sigma^{2} &= Var\left[\sqrt n (\hat{\lambda} - \lambda)\right]\\ &= n Var (\hat{\lambda})\\ &= n Var \left[ \dfrac{n}{\sum_{i=1}^{n}X_{i}} \right] \\ &= n^{3} Var \left[ \dfrac{1}{\sum_{i=1}^{n}X_{i}}\right] \end{align}

But I don't know how to proceed from here. I think the sum of exponentials follows a gamma with parameters $(n,\lambda)$, and that $\dfrac{1}{\sum_{i=1}^{n}X_{i}}$ then follows an inverse gamma?

Answer 1

Yes you are almost there.

You can calculate the variance of $\hat{\lambda}=\frac{n}{Y}$ remembering that

$$Y=\Sigma_i X_i \sim \text{Inverse Gamma}$$

and thus you immediately solve the problem as its variance is a known parameter

...but you can also solve the problem using the integration, and calculating

$$\mathbb{V}\Bigg[\frac{1}{Y}\Bigg]=\mathbb{E}\Bigg[\frac{1}{Y^2}\Bigg]-\mathbb{E}^2\Bigg[\frac{1}{Y}\Bigg]$$

It is not difficult:

$$\mathbb{E}\Bigg[\frac{1}{Y}\Bigg]=\int_0^{\infty}\frac{1}{y}\frac{\lambda^n}{\Gamma(n)}y^{n-1}e^{-\lambda y}dy=\frac{\lambda}{n-1}\underbrace{\int_0^{\infty}\frac{\lambda^{n-1}}{\Gamma(n-1)}y^{(n-1)-1}e^{-\lambda y}dy}_{=1}=\frac{\lambda}{n-1}$$

and similarly for the second simple moment

Anyway this is not the asymptotic variance but it is the exact variance. To calculate the asymptotic variance you can use Delta Method

---

After simple calculations you will find that the asymptotic variance is $\frac{\lambda^2}{n}$ while the exact one is $\lambda^2\frac{n^2}{(n-1)^2(n-2)}$

Answer 2

Following the discussion with @tommik, I am still struggling with something:

Since $\bar{X}$ is the sample mean and given the properties of $(X_{1},....X_{n})$ in the problem set, the Central Limit Theorem tell us that:

$\sqrt{n}\left(\bar{X} - \dfrac{1}{\lambda}\right) \stackrel{D}{\rightarrow} \mathcal{N}\left(0, \dfrac{1}{\lambda^{2}}\right)$

We can apply the Delta Method:

Consider the function $h(x) = x^{-1}$, where $h'(x)= -x^{-2}$, therefore:

$\sqrt{n}\left(h(\bar{X}) - h(\frac{1}{\lambda}) \right) \stackrel{D}{\rightarrow} \mathcal{N}\left(0, \overbrace{\left[h'(\frac{1}{\lambda})\right]^{2} \dfrac{1}{\lambda^{2}}}^{\sigma^{2}}\right)$

$\sqrt{n}\left(\hat{\lambda} - \lambda\right) \stackrel{D}{\rightarrow} \mathcal{N}\left(0, \left[h'(\frac{1}{\lambda})\right]^{2} \dfrac{1}{\lambda^{2}}\right)$

Where notice that the argument inside of the function $h'()$ is $\dfrac{1}{\lambda} = \lambda^{-1}$.

Let's find out what the variance $\sigma^{2}$ is:

\begin{align*} \sigma^{2} &= \left[h'\left(\frac{1}{\lambda}\right)\right]^{2} \dfrac{1}{\lambda^{2}}\\ &= \left[ - \left(\dfrac{1}{\lambda}\right)^{-2} \right]^{2} \dfrac{1}{\lambda^{2}}\\ &= \dfrac{\lambda^{4}}{\lambda^{2}}\\ &= \lambda^{2}\\ \end{align*}

So, I don't know how to conciliate the comments and answers above.