Using Conditional Expectation to find probability of success for Bernoulli Trials

Question

Here is a homework exercise I am struggling with.

Suppose that $X_1,X_2,...$ are independent random variables with Bernoulli distribution, $P\{X_j =1\}=1−P\{X_j =0\}=p$ where p is unknown. Let us assume that p is random and chosen from the uniform distribution on [0, 1]. Find $E[p | F_n], E[X_{n+1} | F_n]$, where $F_n$ denotes the information in $X_1, . . . , X_n$.

My attempt: Let $S_n = X_1 + ... + X_n.$ I think $E[p | F_n] = E[p|S_n] = \int_0^1p\frac{P(p \textrm{ and } S_n=k)}{P(S_n=k)}dp$. From here we can find that $P(S_n=k)=\int_0^1P(S_n=k|p)dp = \int_0^1 {{n}\choose{k}}p^k(1-p)^{n-k} dp = \frac{k!(n-k)!}{(n+1)!}.$ But, I'm not sure how to find $P(p \textrm{ and } S_n=k)$ so I have not gotten any further.

Answer 1

Tip: Do not use $p$ as both the random variable and the integration token.

---

You have that $S_n\mid p\sim\mathcal{Bin}(n,p)$ and $p\sim\mathcal U[0..1]$ So you have the conditional pmf and the pdf:

$$\begin{align}\mathsf P(S_n{\,=\,}s\mid p{\,=\,}t)~&=~\binom n s t^s(1-t)^{n-s}~\mathbf 1_{s\in[[0..n]], t\in[0..1]}\\[1ex]f_p(t) ~&=~ \mathbf 1_{ t\in[0..1]}\end{align}$$

So by the Law of Total Probability: $~\displaystyle\mathsf P(S_n{\,=\,}s)=\int_0^1 \mathsf P(S_n{\,=\,}s\mid p{\,=\,}t)\,f_p(t)\,\mathrm d t$ which you have evaluated .

Likewise $~\displaystyle\mathsf E(p \,\mathbf 1_{S_n=s})=\int_0^1 t\,\mathsf P(S_n{\,=\,}s\mid p{\,=\,}t)\,f_p(t)\,\mathrm d t$, which I'm sure you can evaluate similarly.

Therefore you seek: $$\begin{align}\mathsf E(p\mid S_n{=}s)~&=~\dfrac{\mathsf E(p\,\mathbf 1_{S_n=s})}{\mathsf P(S_n{\,=\,}s)}\\[1ex]&=~\dfrac{\int_0^1 t\,\mathsf P(S_n{\,=\,}s\mid p{\,=\,}t)\,f_p(t)\,\mathrm d t}{\int_0^1 \mathsf P(S_n{\,=\,}s\mid p{\,=\,}t)\,f_p(t)\,\mathrm d t}\\[1ex]&=~\dfrac{\int_0^1 t^{s+1}(1-t)^{n-s}\,\mathrm d t}{\int_0^1 t^{s}(1-t)^{n-s}\,\mathrm d t}\\[1ex]&=~\dfrac{(n+1)!}{s!~(n-s)!}\int_0^1 t^{s+1}(1-t)^{n-s}\,\mathrm d t\\[1ex]&~~\vdots\end{align}$$

---

Alternatively: $$\begin{align}f_{p\mid S_n}(t\mid s) &= \dfrac{\mathsf P(S_n{\,=\,}s\mid p{\,=\,}t)\,f_p(t)}{\mathsf P(S_n{\,=\,}s)}\\&=\dfrac{\mathsf P(S_n{\,=\,}s\mid p{\,=\,}t)\,f_p(t)}{\int_0^1\mathsf P(S_n{\,=\,}s\mid p{\,=\,}t)\,f_p(t)\,\mathrm d t}\end{align}$$

Answer 2

$P(p \textrm{ and } S_n=k)$ is a slight abuse of notation as it is neither a joint density nor a joint probability mass function, but in a sense it is

$$P(p \textrm{ and } S_n=k)=P( S_n=k\mid p)P(p) = {n\choose k}p^k(1-p)^{n-k} \times 1 = {n\choose k}p^k(1-p)^{n-k}$$ and you in fact subconsciously used this to find $P(S_n=k)=\int P(p \textrm{ and } S_n=k) \, dp$.

If you take this through the calculations, you will end up with $E[p | F_n] = E[X_{n+1} | F_n] = \frac{k+1}{n+2}$. Given $F_n$ (noting $S_n$ is a sufficient statistic), the posterior distribution for $p$ is a beta distribution and the predictive distribution for $X_{n+1}$ is a beta-binomial distribution