Comparing two weird estimators

Question

I have this question that I feel is completely out of the blue. We are studying the first subjects in statistics (bias, mean squared error, consistence) and we have been given the task to compare the following estimators for $P(X=0)$ where $X\sim Pois(\lambda)$: $$ \hat{\theta}=e^{-\bar{x}}$$ $$\hat{\zeta}=\frac{\sum_{i=1}^{n}{I(X_{i}=0)}}{n}$$

$I$ being the indicator r.v.

I know sum of Poisson r.v is Poisson, and that I have to use the moment generating function with the first estimator, but I only found the expected value of the second one, and have not been able to get anywhere finding the expected value of the first. Same with the MSE and checking consistency.

Answer 1

It is easy to see that $\mathbb E[\hat\zeta] = \mathbb P(X=0)$ and $\mathbb E[\hat\zeta^2] = \mathbb P(X=0)$. That is, the estimator is unbiased. Thus, the MSE of $\hat\zeta$ is given by $\mathbb n^{-1}P(X=0)-n^{-1}\mathbb P(X=0)^2$. The MSE converges to zero as $n$ approaches infinity. This implies that $\hat\zeta$ converges in probability to $\mathbb P(X=0)$.

Now consider $\hat\theta$. By definition of the Poisson distribution it holds that $$\mathbb E[\hat\theta] = \sum_{k=0}^\infty\hat\theta\frac{(n\lambda)^k}{k!}\exp(-n\lambda) = \sum_{k=0}^\infty\exp(-kn^{-1})\frac{(n\lambda)^k}{k!}\exp(-n\lambda) = \exp(-n\lambda+n\lambda\exp(-n^{-1})).$$ Furthermore, it holds that $\mathbb E[\hat\theta^2] = \exp(-n\lambda + n\lambda\exp(-2n^{-1}))$. This estimate is biased, which makes computing the MSE tedious... Since $\exp$ is continuous and the Poisson distribution has finite second moments, the law of large numbers in conjunction with the continuous mapping theorem shows that $\hat\theta$ converges in probability to $\exp(-\lambda)$ ($=\mathbb P(X=0)$). So $\hat\theta$ is a consistent estimator.

Answer 2

$$E[\hat{\zeta}] = \frac{1}{n} \sum_{i=1}^n E[I(X_i=0)] = E[I(X_1=0)] = P(X_1=0)=e^{-\lambda}$$ so the bias is zero.

$$\text{MSE}(\hat{\zeta}) = \text{Var}(\hat{\zeta}) \overset{?}{=} \frac{1}{n^2} \sum_{i=1}^n \text{Var}(I(X_i=0)) = \frac{1}{n} \text{Var}(I(X_1=0))=\frac{1}{n} e^{-\lambda}(1-e^{-\lambda})$$

The fact that the MSE vanishes as $n \to \infty$ can help you prove consistency (e.g. use Chebychev's inequality).

---

$$E[e^{-\bar{x}}]=E[e^{-X_1/n} e^{-X_2/n} \cdots e^{-X_n/n}] \overset{?}{=} E[e^{-X_1/n}] E[e^{-X_2/n}] \cdots E[e^{-X_n/n}].$$ (Why does the second equality hold?)

Then, use r.e.s.'s hint in the comments to compute each term on the right-hand side. $$E[e^{-X_1/n}] = \sum_{k=0}^\infty e^{-k/n} e^{-\lambda} \frac{\lambda^k}{k!} = e^{-\lambda} \sum_{k=0}^\infty \frac{(\lambda e^{-1/n})^k}{k!} = e^{-\lambda} e^{\lambda e^{-1/n}} = e^{-\lambda (1-e^{-1/n})}.$$ Finally, $$E[e^{-\bar{x}}] = E[e^{-X_1/n}]^n = e^{-\lambda n(1-e^{-1/n})}.$$

Note that $\lim_{n \to \infty} n(1-e^{-1/n}) = -\lim_{x \downarrow 0} \frac{e^{-x}-1}{x} = - \left[\frac{d}{dx} e^{-x}\right]_{x=0} = 1$, so the mean of $\hat{\theta}$ approaches $e^{-\lambda} = P(X=0)$. But for finite $n$, the estimator is biased.

For the variance, let's compute $E[(e^{-\bar{x}})^2]$. By a very similar computation, we have $$E[(e^{-\bar{x}})^2] = E[e^{-2\bar{x}}] = E[e^{-2X_1/n}]^n = e^{-\lambda n(1-e^{-2/n})}.$$ So, $$\text{Var}(e^{-\bar{x}}) = e^{-\lambda n(1-e^{-2/n})} - e^{-2\lambda n(1-e^{-1/n})}.$$

Finally, the MSE is $$\begin{align}E[(e^{-\bar{x}} - e^{-\lambda})^2] &= \text{Var}(e^{-\bar{x}}) + (\text{Bias}(e^{-\bar{x}}))^2 \\ &= e^{-\lambda n(1-e^{-2/n})} - e^{-2\lambda n(1-e^{-1/n})} + (e^{-\lambda n(1-e^{-1/n})} - e^{-\lambda})^2, \end{align}$$ which tends to zero as $n \to \infty$, so the estimator is consistent.