In a survey of 100 people (population=1000), 100 people responded A and 0 people responded B. How to determine margin of error?

Question

I took a few stats classes in university but it's been a decade since I did any of this so I apologize if this is trivial.

I am looking at a research paper and 100 people were surveyed (total population is 1000). All 100 people responded A and 0 people responded B. Trying to determine how to calculate a margin of error for say a 95% (or 99%) confidence level, but things seem confusing (to me lol) when dealing with a completely lopsided response.

Can anyone help? Thank you!

Answer 1

Roughly, the margin of error (for 95% confidence) for a sample of 100 is 10.

But that is for a single variable. In this case, you have two variables ... so figuring out a threshold for statistical significance might make more sense (and that's not simply two times 10, but for 100 in each sample it would be 13

Then again, you don't have two samples here, but just one, and the answers are dependent on each other. That is, I assume that they were given a choice: A or B. So, you can't use your standard statistical significance mesures here either.

Common sense tells you that this result is totally significant though. Whether the difference needs to be 13 or 20, or ... surely this differece of 100 out of 100 should settle things.

Finally, using typical margins of error calculations and statistical significance thresholds only work well when your results are not at the 'boundary' ... as I said, for a sample of 100 you can roughly take a margin of error of 10% ... but if you get 0, the margin is bound to be a bit lower than that. And in this case you clearly are at exactly this border ... so all the more reason to say that this is statistically significant.

Answer 2

Imagine that $n$ people out of the population of $1000$ answered "A". So the proportion of people who answered "A" is:

$$p=\frac{n}{1000}$$

Then, assuming that your sample of 100 was totally random, the probability of selecting $100$ people who answered "A" was:

$$\underbrace{p\times p \times ... \times p}_{100 \, \mathrm{times}} = p^{100}$$

Since $p=\frac{n}{1000}$, we have $$p^{100}=\frac{n^{100}}{1000^{100}}$$

For $95\%$ confidence, set $p^{100}$ equal to $0.05$ and solve the equation:

$$\frac{n^{100}}{1000^{100}}=0.05$$

$$\implies \frac{n}{1000}=\sqrt[100]{0.05}=0.970$$

$$\implies n=970$$

Which means that you can be $95\%$ confident that $n$ (the number of people who responded "A") is greater than $970$.

We would say that $970$ is the "critical value" at a "$5\%$ significance level".

Perhaps try to work through the calculation yourself with $0.01$ instead of $0.05$, to find the critical value at a $1\%$ significance level.

Answer 3

The actual formula for margin of error is:

$$E = z\sqrt{\frac{\widehat p(1-\widehat p)}{n}}$$

So you can see that $1 - \widehat p = 0$, hence $E = 0$ which isn't useful.

This is the problem pointed out in Bram28's response.

However, if I run a one proportion Z test to see what population proportion is statistically significantly different, I get $.97 (p = .039)$ So I would say a reasonable margin of error would be $.02$.

That is, for a sample proportion of 1, it is reasonable to assume a margin of error of $.02$ whereby a population proportion of $.98$ is not statistically significantly different than $1$ where $(p >.05) (95\%$ significance).