Before posing this question, the lecture notes I am reading discussed games, probability, the binomial distribution and central limit theorem. It usually assumes some form of game when it asks something. This question has been confusing me a bit:
How many experiments do we need to perform to estimate with $90$% confidence a winning probability within an accuracy of $1$%? And if we want an accuracy of $0.1$%?
We want to estimate $p$, let us use $\hat{p}$. Normally the confidence interval for $90$% certainty for a binomial distribution (win/lose) is given by: $$(\hat{p}-\frac{ 1.645 \sqrt{\hat{p}(1-\hat{p})}}{\sqrt{N}},\hat{p}+\frac{ 1.645 \sqrt{\hat{p}(1-\hat{p})}}{\sqrt{N}} ) $$
Here we have that the last bit is the uncertainty, we want this to be equal to $1$% so $0.01$, we then get that: $$ 0.01=\frac{1.645 \sqrt{\hat{p}(1-\hat{p})}}{\sqrt{N}} $$
We get that we should at least pick: $$ N \geq 1.645^2\hat{p}(1-\hat{p}) \cdot 10^4$$
Is it true that I cannot directly compute $\hat{p}$ from how the answer is phrased and this would be the best answer?
The answer for the second question will be: $$ N \geq 1.645^2\hat{p}(1-\hat{p}) \cdot 10^6$$
By symmetry of the zeros of $f(x)=x(1-x)$ at $x=0$ and $x=1$, we know that this estimate for $N$ is maximised whenever $\hat{p}= \frac{1}{2}$. This would indeed correspond to the worst case scenario for an estimator. $$ N_1 \geq 1.645^2 \cdot \frac{1}{4} \cdot 10^4 \approx 6765 $$ And similarly: $$ N_2 \geq 1.645^2 \cdot \frac{1}{4} \cdot 10^6 \approx 676506 $$
By symmetry of the zeros of $f(x)=x(1-x)$, we know that this estimate for $N$ is maximised whenever $\hat{p}= \frac{1}{2}$ $$ N_1 \geq 1.645^2 \cdot \frac{1}{4} \cdot 10^4 \approx 6765 $$ And similarly: $$ N_2 \geq 1.645^2 \cdot \frac{1}{4} \cdot 10^6 \approx 676506 $$