interview question of hypothesis test: how many tests are needed to tell the difference between an unfair coin (p=0.49) and a fair coin (p=0.5)

Question

Here is an interview question of hypothesis test:

how many tests are needed to tell the difference between an unfair coin ($p_1=0.49$) and a fair coin ($p_2=0.5$)?

My idea is constructing the normal sample to use Independent two-sample t-test.

We denote the Bernoulli distribution of k-th coin (k = 1, 2) for i-th test: $T_k(i)\sim Bernoulli(p_k).$ Then

$$B_k(i)=\dfrac{\sum_{i=1}^nT_k(i)}{n}\sim N\left(p_k,\dfrac{p_k(1-p_k)}{n}\right).$$

However $B_k(i)$ is not a i.i.d. sample (variance changes), which prevents us from using independent two-sample t-test.

Edit:

I think the question is not clear, however for my opinion, it hopes us to test the difference of two binomial means. Therefore I think we should use Independent two sample Z test for binomial samples: $$H_0: p_1 = p_2 = p,$$ $$Z = \dfrac{\hat{p}_1-\hat{p}_2}{\hat{p}(1-\hat{p})\left(\dfrac{1}{N_1}+\dfrac{1}{N_2}\right)}.$$

And the value of $N_1,N_2$ (we can simply assume $N_1=N_2=N$) depends on the confident level.

Answer 1

Quick and dirty answer, which is what many interview questions seek:

The width of a 95% CI for $p$ is $1.96\sqrt{p(1-p)/n} \approx \sqrt{1/n}$ in close two-person races. So in public opinion polling $n = 2500$ subjects gives margin of error about $\pm 2\%. $

Here margin of error is about 1%, so $n\approx 10\,000.$

Somewhat more formally, one might insist on using sample size large enough to reject $H_0: p = 0.49$ against $H_a: p > 0.49$ for specific alternative $p = 0.50.$ It turns out that $30\,000$ suffices to get good power, as shown by the following simulation in R:

set.seed(919)
pv = replicate(10^5, 
  prop.test(rbinom(1,30000,.49), 30000, alt="less")$p.val)
mean(pv <= .05)
[1] 0.96614      $ aprx power of test