Suppose you have two possible outcomes, $0$ or $1$. You want to determine the probability of getting a $1$, and have confidence intervals on your answer.
E.g. for $10$ runs you might have
$$ \text{experimental results} = 0, 1, 1, 1, 1, 1, 1, 1, 1, 1.$$
Directly from the results you could calculate the probability (mean) of getting a $1$, $pr(1) = 0.9$.
For the confidence interval, how would you calculate the standard deviation or standard error?
Would it be based on the raw data? E.g. in Excel $\text{stdev}(0, 1, 1, 1, 1, 1, 1, 1, 1, 1)$
Or based on the running mean (probability) of the responses. E.g. $\text{stdev}(0, 0.5, 0.667, ..., 0.9)$
I believe it should be based on the variation in the running mean as this is the probability I am interested in, but I'm having a mental blank on trying to justify it. If I wasn't interested in the probability I would just calculate the standard error directly from the results, not after taking the mean - the extra step is throwing me off.