Let $X_1...X_n\sim\rm{ Bernoulli}(\theta)$. Show that the variance of the MLE attains the Cramer-Rao lower bound.

Question

I know that the MLE for $\theta$ is the sample mean:

$$\hat \theta = \frac{1}{n}\sum_{i=1}^nX_i$$

and that \begin{align} Var\left(\frac{1}{n}\sum_{i=1}^nX_i\right) &= \frac{1}{n^2}\sum_{i=1}^n Var(X_i) \\ &= \frac{1}{n^2}\sum_{i=1}^n \theta(1-\theta) \\ &= \frac{n}{n^2}\theta(1-\theta) \\ &= \frac{1}{n}\theta(1-\theta) \\ &= \frac{\theta(1-\theta)}{n} \\ \end{align}

I also know that this equality should hold:

\begin{align} Var\left(\frac{1}{n}\sum_{i=1}^nX_i\right) &= \frac{\left(\frac{d}{d\theta}E\left[\left(\frac{1}{n}\sum_{i=1}^nX_i\right)\right]\right)^2}{E\left[\left(\frac{d}{d\theta} \ln\left(f(X\mid\theta)\right)\right)^2\right]} \\ &= \frac{1}{E\left[\left(\frac{\sum_{i=1}^n X_i}{\theta} -\frac{n - \sum_{i=1}^n X_i}{1-\theta}\right)^2\right]} \\ \end{align}

Can someone show me that $E\left[\left(\frac{\sum_{i=1}^n X_i}{\theta} -\frac{n - \sum_{i=1}^n X_i}{1-\theta}\right)^2\right] = \frac{n}{\theta(1-\theta)}$?

I just don't see it. Someone help me here.

Answer 1

Let $$n \bar x = \sum_{i=1}^n X_i.$$ Then $$\frac{n \bar x}{\theta} - \frac{n - n \bar x}{1 - \theta} = \frac{n}{\theta(1-\theta)} (\bar x - \theta).$$ Consequently, $$\operatorname{E}\left[\left(\frac{n \bar x}{\theta} - \frac{n - n \bar x}{1 - \theta}\right)^2\right] = \frac{n^2}{(\theta(1-\theta))^2} \operatorname{E}[(\bar x - \theta)^2]. $$ Since you already know $$\operatorname{E}[\bar x] = \theta,$$ then $$\operatorname{E}[(\bar x - \theta)^2] = \operatorname{Var}[\bar x] = \frac{\theta(1-\theta)}{n}$$ and the rest is straightforward.