Determine the maximal likelihood estimator for the coin probability

Question

I have $X_n$ - number of heads after $n$ coin tosses with $X_n\mid P=p \sim \operatorname{Bin}(n,p) $ and $P \sim \operatorname{U}(0,1)$

I need to find a) $f_{P\mid X_3=2}(p)$, the conditional density function of P given $X_3=2$ and b) determine the maximal likelihood estimator for the probability of getting a head in one coin toss.

I found first the general case of of $f_{P\mid X_n=k}(p)$ via via posterior distribution of $P$ :

$$P(P\le x\mid X_n=k)=\frac{\int_0^xP(X_n=k\mid P=y)\cdot f_P(y)\,dy}{P(X_n=k)}= \frac{\int_0^x{n \choose k}y^k(1-y)^{n-k}\cdot 1 \, dy}{\frac{1}{n+1}}= (n+1)\frac{n!}{k!(n-k)!}\int_0^xy^k(1-y)^{n-k} \, dy.$$ Using Gamma and differentiating, we have:

$$f_{P\mid X_n=k}(p)=\frac{(n+2-1)!}{(k+1-1)!(n+1-k-1)!}\cdot x^k(1-x)^{n-k} \\ = \frac{\Gamma(n+2)}{\Gamma(k+1)\Gamma(n+1-k)} \cdot x^k(1-x)^{n-k},$$ so with $X_3=2$, we have:

$$f_{P\mid X_3=2}(p)=\frac{\Gamma(3+2)}{\Gamma(2+1)\Gamma(3+1-2)} \cdot x^2(1-x)^{3-2} = 12x^2(1-x)$$

a) Did I solve correctly the part a) ?

b) I am not sure how to do part b)?

Thank you very much for any help or feedback!

Answer 1

Part $(a)$ is correctly solved, except that you call the same variable both $x$ and $p.$ You should end up saying $$ f_{P\,\mid\, X_3=2} (p) = 12 p^2(1-p) \quad\text{for } 0\le p\le 1. $$

For part $(b)$ you need to consider $$ L(p) = \Pr(X_3=2) = \text{constant} \times p^2(1-p) $$ and then $$ \ell(p) = \log L(p) = \text{constant} + 2\log p + \log(1-p). $$ Then $$ \ell\,'(p) = \frac 2 p - \frac 1 {1-p}. $$ Set this equal to $0$ and solve for $p.$ That's not enough by itself to prove there's a unique absolute maximum at that point, but you have these additional facts:

• $L(p) = 0$ if $p=0$ or $p=1,$ and
• $L(p) > 0$ if $0<p<1,$ and
• $L(p)$ is an everywhere differentiable function of $p,$ and
• therefore an absolute maximum point exists, and the derivative is $0$ at that point, so
• if there is only one point in the interior of the interval $[0,1]$ at which the derivative is $0,$ then that's it.

Answer 2

The computation can be simplified by first observing $$f_{P \mid X_3} (p \mid x) = \frac{\Pr[X_3 = x \mid p]f_P(p)}{\Pr[X_3 = x]} = \frac{\Pr[X_3 = x \mid p]}{\Pr[X_3 = x]} = \frac{\binom{3}{x}p^x (1-p)^{3-x}}{\Pr[X_3 = x]}.$$ But since the denominator, as well as the binomial coefficient $\binom{3}{x}$, are independent of $p$, the posterior density of $P$ is simply proportional to $$f_{P \mid X_3}(p \mid x) \propto p^x (1-p)^{3-x}.$$ It is now trivial to observe that the desired constant of proportionality must equal $$\left(\int_{p=0}^1 p^x (1-p)^{3-x} \, dp \right)^{-1}.$$ Since we observed $X_3 = 2$, we simply integrate $$\int_{p=0}^1 p^2 (1-p) \, dp = \left[\frac{p^3}{3} - \frac{p^4}{4}\right]_{p=0}^1 = \frac{1}{3} - \frac{1}{4} = \frac{1}{12},$$ thus the desired constant is $12$ and the posterior density is $$f_{P \mid X_3 = 2}(p) = 12p^2 (1-p), \quad 0 < p < 1.$$

For the second part, what is the probability of getting a head in a single toss? It is $$\Pr[X_1 = 1] = \binom{1}{1} p^1 (1-p)^{1-1} = p.$$ So the question simply asks you for the MLE of $p$.