Tossing two coins with unknown success probability

Question

I have two coins with uniformly distributed unknown success probabilities $p_1$ and $p_2$.

I toss both coins $N$ times and got $n_1$ and $n_2$ successes for the first and second coins respectively. ($N$ is large, both $n$s are small. $N \gg n_1 > n_2$).

Knowing this, what is the probability that $p_1 > p_2$.

I understand that it is a conditional probability and I think that I can solve for $N=1$, $n_1=1$ and $n_2=0$. [at least calculations in my head claim this is $5/6$].

I suppose that I need to integrate probabilities of getting the outcome on the lower triangle vs integrating on the whole square.

Since, this approach did not give me anything intuitive or simple (to be solved on the paper),

Is there anything simpler, and/or more intuitive?

Update:

After I have been asked few times, how to calculate things in the head...

First of, deprive yourself of paper or access to the internet. Calculations involved in this case are not too difficult and, when you lack any tool, you can give a try.

So we have to calculate: $$\frac{\int\limits_0^1\int\limits_{p_2}^1 p_1(1-p_2) dp_1 dp_2}{\int\limits_0^1\int\limits_{0}^1 p_1(1-p_2) dp_1 dp_2}.$$

The upper integral is much more involved, but in the end it is a polynomial, and you solve it like on the paper:

$$I = \int\limits_0^1\int\limits_{p_2}^1 p_1(1-p_2) dp_1 dp_2 = \int\limits_0^1 \frac{1}{2}p_1^2\Big|_{p_2}^1(1-p_2) dp_2, $$ then $$I = \frac{1}{2}\int\limits_0^1 (1-p_2^2)(1-p_2)dp_2.$$

Here is the point where you should believe in yourself, specifically, that you can hold $4$ numbers (simple monomials) in the head at once, cached.

$$I = \frac{1}{2}\int\limits_0^1 (1-p_2-p_2^2+p_2^3)dp_2.$$

But luckily, $\int\limits_0^1 x^k = \frac{1}{k+1},$ so we have that $$I = \frac{1}{2} \cdot \left[\frac{1}{0+1} - \frac{1}{1+1} - \frac{1}{2+1}+\frac{1}{3+1}\right] = \frac{5}{24}$$

The second integral, can be calculated directly, or we can notice, that due to symmetry(ies) it is exactly $\frac{1}{4}$.

Answer 1

In general, assuming uniform $B(1,1)$ priors on $p_1$ and $p_2$, you have $$f_X(x) = \text{PDF of }B(1 + n_1, 1 + (N-n_1)),$$ $$f_Y(y) = \text{PDF of }B(1 + n_2, 1 + (N-n_2)).$$ So the PDF of $Z = X - Y$ is

$$f_Z(z) = \int f_Y(y)f_X(y+z)dy,$$

(because $X=Y+Z$) where the integration range is $(0,1-z)$ for $z>0$ and $(-z,1)$ for $z<0$. And then simply $\mathbb{P}(Z>0)=\int_0^1f_Z(z)dz$.

I applied this general method to verify your calculation that for $N=1,n_1=1,n_2=0$, the answer is $5/6$, by calculating

$$f_Z(z)=\left\{ \begin{align} &\frac23 (1-z)(1+4z+z^2) &\text{ if } z>0\\ &\frac23 (z+1)^3 &\text{ if } z<0 \\ \end{align} \right.$$

How did you do it in your head?

Answer 2

The probability is not hard to compute. I will be using the same argument as here. Simulate, all independently,

$$ \begin{align} X_1, X_2, \dots, X_{N+1} &\sim \text{Unif}[0,1), \quad p_1:=X_1,\\ Y_1, Y_2, \dots, Y_{N+1} &\sim \text{Unif}[0,1), \quad p_2:=Y_1.\\ \end{align} $$

Note that for $i\neq 1$, letting $A_i=\mathbf{1}\{X_i<X_1\}$, we have $\mathbb{P}(A_i)=p_1$, so we can interpret these $X_i$'s (and similarly $Y_i$'s) as the coin tosses.

You can see the $[0,1)$ segment partitioned into $N+2$ small segments $\left[X_{(i)}, X_{(i+1)}\right)$, where $X_{(i)}$ is the $i$th order statistic, and $X_{(0)}=0, X_{(N+2)}=1$. The problem requires $X_1 = X_{(n_1+1)}$, $Y_1 = Y_{(n_2+1)}$, and asks us to find $\mathbb{P}(Y_1 \le X_1)$. So we count the number of ways to add the $Y_i$ to these segments in two cases: $A$: satisfying $Y_1 \le X_1$, and $B$: without this restriction.

Note that if you put $Y_1$ in the $k$th segment (i.e. $Y_1\in[X_{(k-1)},X_{(k)})$), then $[0,1)$ is now partitioned into $N+3$ small segments, with $k$ segments before $Y_1$ and $N+2-(k-1)$ segments after. To satisfy $Y_1 = Y_{(n_2+1)}$, you need to put $n_2$ of $Y_i$ before $Y_1$ and $N-n_2$ after it. So in total there are $k^{n_2}(N+3-k)^{N-n_2}$ ways for case $B$.

For case $A$, note that $Y_1\le X_1$ is simply $Y_1\le X_{(n_1+1)}$, so $Y_1$ is in the $k$th segment where $k\le n_1+1$. So

\begin{align} \mathbb{P}(Y_1\le X_1)=&\frac{A}{B}\\ =&\frac{\sum_{k=1}^{n_1+1}\text{num. ways when } Y_1\in[X_{(k-1)},X_{(k)})} {\sum_{k=1}^{N+2}\text{num. ways when } Y_1\in[X_{(k-1)},X_{(k)})}\\ =&\frac{\sum_{k=1}^{n_1 + 1} k^{n_2} (N+3-k)^{N-n_2}} {\sum_{k=1}^{N + 2} k^{n_2} (N+3-k)^{N-n_2}}.\\ \end{align}

Indeed, plugging in $N=1,n_1=1,n_2=0$ gives $$\frac{1^03^1+2^02^1}{1^03^1+2^02^1+3^01^1}=\frac{3+2}{3+2+1}=\frac56.$$