I wrote a simulation that does 1,000,000 trials. It returns 430,200 successes. I want to calculate a probability of success with confidence interval for that probability estimate.
I use $\sigma = \sqrt {np(1-p)} = \sqrt{1e6*0.4302*(1-0.4302)} \approx$ 495.1.
Then the probability of success $\approx$ 0.4302 $\pm$ 0.00097, p=0.05.
Is that correct?
Seems weird to me. Seems like the formula $$ \sqrt{np(1-p)} $$ should be used for a known probability of success and then used to calculate the standard deviation.
In other words, what I want to know is can that same formula be used to go the other direction? That is, start with a simulated probability of success and use that along with the number of trials to derive the estimated probability of success with confidence interval.
For large $n,$ we have $ \frac{\hat{p} - p}{\sqrt{p(1-p)/n}} \sim N(0,1),$ where $\hat{p}$ is the sampled (simulated) success rate. Therefore the $100(1-\alpha)$% confidence interval is $$\hat{p} \pm z_{\alpha/2} \sqrt{p(1-p)/n}.$$ If you know what the exact value of $p$ is (rare in practice), then you should you the CI above. Otherwise, you should use $$ \hat{p} \pm z_{\alpha/2} \sqrt{\hat{p}(1-\hat{p})/n}. $$