Low Number of trials for Bernoulli trial - cannot use normal approximation

Question

I am investigating whether an event that can be treated as a Bernoulli trial is statistically likely. However, I only have $ n = 5 $ trials, and so I don't believe I can approximate a normal distribution and employ a hypothesis test. Is there any way around this? I understand that statistics inherently relies on the law of large numbers so I assume I just have too few trials ultimately.

Answer 1

You are correct not to try using a normal approximation in these circumstances. However, you can do an exact binomial test.

Suppose you want to test $H_0: p = 0.8$ vs. $H_a: p < 0.8.$ If you have $n = 5$ Bernoulli trials, then according to $H_0$ the number of Successes $X \sim \mathsf{Binom}(n=5, p=.8).

Now suppose you get $X = 2$ successes. Assuming $H_0$ to be true, what is the probability $P(X \le 2)?$ You can find $$P(X \le 1) = P(X=1)+P(X=1) = 0.00672$$ by applying the binomial PDF $$P(X=k) = {n\choose k}p^k(1-p)^{n-k},$$ for $k = 0,1.$ Or you can use software to get the answer, as in R below (where pbinom is a binomial CDF).

pbinom(1, 5, .8)
[1] 0.00672

Thus the P-value for your one sided test is $0.00672 < 0.05 = 5\%,$ and you can reject $H_0$ at the 5% level.

If you were to observe $X = 3,$ then the P-value would be $P(X \le 3) =0.05792$ and you could not reject at the 5% level. (But you could, just barely, reject at the 6% level.)

pbinom(2, 5, .8)
[1] 0.05792