How to find UMVUE of $\theta^k$ when $x_1, \ldots, x_n$ is a sample from Bernoulli$(\theta)$?

Question

Let $x_1, x_2, \ldots, x_n$ be a random sample from the Bernoulli ($\theta$).

The question is to find the UMVUE of $\theta^k$.

I know the $\sum_1^nx_i$ is the complete sufficient statistics for $\theta$.

Is $\left(\frac{\sum_1^nx_i}{n}\right)^k$ the estimator or any other possible estimator?

Could someone just help me?

Answer 1

Having that

$$\theta^m=P\{ X_1=x_1,X_2=x_2,...,X_m=x_m\}$$

An unbiased estimator for $\theta^m$ is

$$T= \begin{cases} 1, & if \ \ X_1=X_2= \, ... \,=X_m =1 \\ 0, & in \ other \ case \end{cases}$$

But $$\begin{align} E[T|S=s] & = P\{X_1=1,X_2=1,...,X_m=1|S=s\}=\frac{P\{X_1=1,X_2=1,...,X_m=1,S=s\}}{P\{S=s\}} = \\\\ & = \begin{cases} 0, & if \ \ m>s \\ \frac{\theta^m\binom{n-m}{s-m}\theta^{s-m}(1-\theta)^{n-s}}{\binom{n}{s}\theta^s(1-\theta)^{n-s}}, & if \ \ m\leq s \end{cases} \end{align}$$

By the theorem of Lehmann-Scheffé, the UMVUE for $\theta^m$ is, after operating the latter expression:

$$E[T|S=s]=\begin{cases} 0, & if \ \ m>s \\ \frac{s!(n-m)!}{n!(s-m)!}, & if \ \ m\leq s \end{cases}$$

Answer 2

• You have shown that $T=\sum\limits_{i=1}^n X_i$ is a complete sufficient statistic for $\theta$.

• It is easy to verify that $E_{\theta}\left(\frac{T}{n}\right)=\theta$ and $E_{\theta}\left(\frac{T(T-1)}{n(n-1)}\right)=\theta^2$ for all $\theta\in(0,1)$.

• Inductively or otherwise one can show that

$$E_{\theta}\left(\frac{T_{(k)}}{n_{(k)}}\right)=E_{\theta}\left(\frac{T(T-1)(T-2)\ldots(T-k+1)}{n(n-1)(n-2)\ldots(n-k+1)}\right)=\theta^k$$ for all $\theta\in(0,1)$ and for integer $k$ provided $0< k\le T\le n$.

Since $T\sim \mathsf{Bin}(n,\theta)$, we can also take some function $g$ of $T$ so that $g(T)$ is unbiased for $\theta^k$:

$$E_{\theta}\left[g(T)\right]=\sum_{j=0}^n g(j)\binom{n}{j}\theta^j(1-\theta)^{n-j} =\theta^k$$

Comparing coefficients we can solve for $g(\cdot)$.

And the direct method outlined by @Akerbeltz is essentially the application of Rao-Blackwell theorem to find the UMVUE as $E\left[\prod\limits_{i=1}^k X_i\mid T\right]$.

Being a function of the complete sufficient statistic, $\frac{T_{(k)}}{n_{(k)}}$ is the UMVUE of $\theta^k$ by Lehmann-Scheffe theorem whenever $k$ is an integer with $0<k\le T\le n$.