Mean Squared Error (MSE) of parameter $p$ in binomial distribution: intuition

Question

Let $X \sim Bin(n, p)$ and let

$$\hat p = \frac{X}{n} $$ be an estimator of the probability $(p)$ of success in this binomial distribution, where $X$ is the number of successes and $n$ is the number of trials.

My question is, why does $\text{MSE}(\hat p)\rightarrow 0$ as $p \rightarrow0 \, \text{ or }\, 1$?

Answer 1

Since $X\sim Bin(n,p)$, then $\mathbb E[X]=np$ and $\mathbb E[X^2]=np((n-1)p+1)$, where $\mathbb E[X^2]$ can be deduced from the well known fact $Var(X)=np(1-p)=\mathbb E[X^2]-\mathbb E[X]^2$. Hence \begin{align} \operatorname {MSE} \left(\hat p\right)&=\mathbb {E} \left[\left(\hat p -p \right)^{2}\right]=\mathbb E\left[\left(\frac{X}{n}-p\right)^2\right]=\mathbb E\left[\frac{X^2}{n^2}-\frac{2pX}n+p^2\right]\\[0.3cm]&=\frac1{n^2}\mathbb E\left[X^2\right]-\frac{2p}{n}\mathbb E[X]+p^2\\[0.3cm]&=\frac{np}{n^2}\left((n-1)p+1\right)-\frac{2}{n}np^2+p^2\\[0.3cm]&=\frac{p(1-p)}{n}\longrightarrow0, \qquad \text{ for}\,\, p\to\{ 0,1\}\end{align}

Intuitively, if $p\to 0$ or $1$, then the uncertainty in your experiment tends to vanish: your sample will either have only successes $(p=1)$ or only failures $(p=0)$. Hence, your estimation will have great accuracy. To gain more understanding, consider the case where uncertainty of the outcome of the experiment is maximized, i.e. when $p=1/2$. In this case $p(1-p)$ is maximum hence also the MSE.

Answer 2

The above answer (from Jimmy R.) is correct. However you missed a $p$ term while squaring.

The answer then becomes:

\begin{align} \operatorname {MSE} \left(\hat p\right)&=\mathbb {E} \left[\left(\hat p -p \right)^{2}\right]=\mathbb E\left[\left(\frac{X}{n}-p\right)^2\right]=\mathbb E\left[\frac{X^2}{n^2}-\frac{2Xp}n+p^2\right]\\[0.3cm]&=\frac1{n^2}\mathbb E\left[X^2\right]-\frac{2}{n}\mathbb E[X]p+p^2\\[0.3cm]&=\frac{np((n-1)p+1)}{n^2} - \frac{2}{n}np^2 + p^2\\[0.3cm]&=p^2 - \frac{p^2}{n} + \frac{p}{n} - 2p^2 + p^2\\[0.3cm]&=\frac{p}{n} - \frac{p^2 }{n} = \frac{p(1-p)}{n}\longrightarrow0, \qquad \text{ for}\,\, p\to\{ 0,1\}\end{align}

All of the remaining conclusions are still valid!

Also note you can write it as $\operatorname {MSE} \left(\hat p\right)=\mathbb {E} \left[\left(\hat p -p \right)^{2}\right] = \text{Var}(\hat p) + \left(\mathbb {E} \left(\hat p\right) -p\right)^2$ which can be easier to solve, as $\mathbb {E} \left(\hat p\right) = p$ and so the 2nd term of the sum is null.