Let $X_1, X_2,X_3\sim\rm{ Bernoulli}(\theta)$. Show that $I_{X_1+X_2>X_3}$ is an unbiased estimator of $h(\theta)$ and find UMVUE of $h(\theta)$

Question

Let $X_1, X_2,X_3\sim\rm{ Bernoulli}(\theta)$. Show that $I_{X_1+X_2>X_3}$ is an unbiased estimator of $h(\theta)=P_{\theta}(X_1+X_2>X_3)$ and find UMVUE of $h(\theta)$.

Assuming $A = (X_1+X_2 > X_3)$, is it correct that $E[I_A]= P(A)$ in this situation?

If this is correct, then I know that for the estimator to be unbiased, $E[I_A] = P(A) = h(\theta)$, which checks out.

I also know that $I_A$ is the best unbiased estimator of $h(\theta)$ if it attains the Cramer-Rao lower bound:

$$\operatorname{Var}\left(I_A\right) \geq \frac{\left(\frac{d}{d\theta}E[I_A]\right)^2}{E\left[\left(\frac{d}{d\theta} \ln\left(f(X\mid\theta)\right)\right)^2\right]}$$

When I try solving for $\operatorname{Var}\left(I_A\right)$:

\begin{align} \operatorname{Var}\left(I_A\right) &= E[(I_A - E[I_A])^2] \\ &= E[(I_A - h(\theta))^2] \\ \end{align}

I get stuck because I don't know how to simplify further (i.e. I don't know how to evaluate $h(\theta)$).

Trying to solve for the left-hand side, I am also stuck because I don't know the joint pmf $f(X\mid\theta)$ that gives rise to $h(\theta)$ when integrated.

Edit:

Because $I_A$ is not the UMVUE, I need to find the best unbiased estimator of $h(\theta)$. Toward this end, I figured out that the joint pmf is $$f(X\mid\theta) = \theta^{(X_1+X_2+X_3)}(1-\theta)^{(3-(X_1+X_2+X_3))}$$ and $$h(\theta) = \theta^2 + 2 \theta (1-\theta)^2$$ but how do I use this information to obtain an estimator $\delta$ such that $E[\delta] = h(\theta)$?

Answer 1

The expected value of an indicator function is the probability that it is equal to $1.$ Thus it is $\Pr(X_1+X_2=1 \text{ or } X_3=0).$ That is enough to show that the indicator function is an unbiased estimator of that probability.

Now assuming $X_1,X_2,X_3$ are independent, one can show that the conditional distribution of $(X_1,X_2,X_3)$ given $X_1+X_2+X_3$ does not depend on $\theta.$ Thus the conditional expected value of the unbiased estimator given $X_1+X_2+X_3$ should give you an improved estimator.

We have \begin{align} & \Pr(X_1+X_2>X_3\mid X_1+X_2+X_3=x) \\[8pt] = {} & \begin{cases} 1 & \text{if } x = 3 \\ 1/3 & \text{if } x = 2 \\ 2/3 & \text{if } x = 1 \\ 0 & \text{if } x = 0 \end{cases} \tag 1 \end{align} Thus the function of $x$ defined by line $(1)$ above, when evaluated at $X_1+X_2+X_3,$ gives you an improved estimator. And here I am actually surprised by the non-monotone nature of this function.

This gives you the best unbiased estimator if the statistic $X_1+X_2+X_3$ is complete. We have \begin{align} & \operatorname E(g(X_1+X_2+X_3)) \\[10pt] = {} & \theta^3 g(3) + 3\theta^2(1-\theta)g(2) \\[3pt] & {} + 3\theta(1-\theta)^2 g(1) + (1-\theta)^3 g(0) \\[10pt] = {} & \big( g(3) -3g(2) + 3g(1) - g(0) \big)\theta^3 \\[3pt] & {} + \big( 3g(2) - 6 g(1) + 3g(0) \big) \theta^2 \\[3pt] & {} + \big( 3g(1) - 3g(0) \big) \theta \\[3pt] & {} + \big( g(0) \big). \end{align} This can remain equal to $0$ as $\theta$ changes only if \begin{align} g(3) -3g(2) + 3g(1) - g(0) & = 0, \\[4pt] 3g(2) - 6 g(1) + 3g(0) & = 0, \\[4pt] 3g(1) - 3g(0) & = 0, \\[4pt] g(0) & = 0. \end{align} This holds only if $g(3)=g(2)=g(1)=g(0)=0.$

Thus we have completeness.

Finally, the Lehmann–Scheffé theorem says we have the UMVUE.

Answer 2

Assuming independence of $X_1,X_2,X_3$, the statistic $T=X_1+X_2+X_3$ is complete sufficient for $\theta$ since the parent distribution is a member of the exponential family.

An unbiased estimator of $h(\theta)$ based on $T$ is the UMVUE of $h(\theta)$ by Lehmann-Scheffe theorem.

Now as you have found, $h(\theta)=2\color{blue}{\theta^3}-3\color{green}{\theta^2}+2\color{red}{\theta}$.

Using the form of the UMVUE of $\theta^k$ for $k\in\mathbb N$, the UMVUE of $h(\theta)$ is clearly

$$g(T)=2\cdot\color{blue}{\frac{T(T-1)(T-2)}{3\cdot2\cdot1}}-3\cdot\color{green}{\frac{T(T-1)}{3\cdot2}}+2\cdot\color{red}{\frac{T}{3}}$$

That is, $$g(T)=\frac{T^3}{3}-\frac{3T^2}{2}+\frac{11T}{6}$$