Example of using Delta Method

Question

Let $\hat p$ be the proportion of successes in $n$ independent Bernoulli trials each having probability $p$ of success. (a) Compute the expectation of $\hat p (1-\hat p)$. (b) Compute the approximate mean and variance of $\hat p (1-\hat p)$ using the Delta Method.

For part (a), I can calculate the expectation of $\hat p$ but got stuck on the expectation of $\hat p^2$, though I doubt this is a right track.

For part (b), I'm new to Delta method and as for the approximate variance I know I need to calculate the derivative of the function $\hat p (1-\hat p)$, but not quite sure. I appreciate if anyone can provide some answers or some similar examples online.

Answer 1

Let $$ k = #{success in n independent Bernoulli trials}$$ then we have $$ k \sim B(n,p)$$ and $$ E(\hat{p})=E(\frac{k}{n})=\frac{E(k)}{n}=\frac{np}{n}=p $$ $$Var(\hat{p})=Var(\frac{k}{n})=\frac{Var(k)}{n^{2}}=\frac{np(1-p)}{n^{2}}=\frac{p(1-p)}{n}$$ thus $$E(\hat{p}^{2})=Var(\hat{p})+ E(\hat{p})^{2}=\frac{p-(1-n)p^{2}}{n} $$ this answers your first question.$$$$ For the second part,$$ Var(\hat{p}(1-\hat{p}))\approx(1-2\hat{p})^{2}Var(\hat{p})=(1-2\hat{p})^{2}\frac{p-(1-n)p^{2}}{n}$$ and plug in the estimate of $\hat{p}$ to get it.

Answer 2

The second moment of a binomial random variable $X \sim \operatorname{Bin}(n,p)$ is $$E[X^2] = \operatorname{Var}(X) + E[X]^2= np(1-p) + n^2 p^2 = np(1-p+np).$$ So, $$E[\hat{p}^2]=\frac{1}{n^2} E[X^2]=\frac{p(1-p+np)}{n}.$$

---

By the central limit theorem, you know $\sqrt{n}(\hat{p}-p) \overset{d}{\to} N(0,p(1-p))$. You want to apply the delta method with $g(\hat{p})=\hat{p}(1-\hat{p})$. The derivative of $g$ is $1-2\hat{p}$, and the square of the derivative evaluated at $p$ is $(1-2p)^2$. Thus, $$\sqrt{n}(\hat{p}(1-\hat{p})-p(1-p)) \overset{d}{\to} N(0,p(1-p)(1-2p)^2).$$