You are on a beach full of rocks looking for treasure. The odds of finding treasure under any rock is $1/273$. You have looked under $5000$ rocks and found only $2$ pieces of treasure. How unlucky were you? What are the odds of looking under $5000$ rocks and finding only $2$ pieces of treasure (when the odds of looking under $1$ rock and finding treasure is $1/273$)? I assume you have to add the probability of finding $0$, $1$ and $2$ together?
I found this website, but the numbers aren't precise enough for me. I would like to convert the odds to something more manageable. I wanted to be able to say, this is equivalent to getting heads X number times in a row for coin tosses. And with my sample data it just turns into "$0.9999999$" on that site, so its not useful to me.
I think I need "Cumulative probability: $P(X ≤ x)$".
If someone could help explain it to me.
Ideally I would also like to know how much outside of the norm my sample set is. In other words, I question the validity of the 1/273 odds and want to know the best way to test it.
"More precise" isn't necessarily the right viewpoint here, especially when you're having issues with a theoretically accurate calculation rounding to $0$ or $1$. Instead, we should look for an approximate model that handles this extreme well.
And on that note - there is such a model. The Poisson distribution arises as a limit of binomial distributions with large numbers of trials, low probability, and fixed mean. That mean becomes the distribution's sole parameter $\lambda$, which will be $\frac{5000}{273}\approx 18.315$ in this case.
In a Poisson distribution with parameter $\lambda$, the probability of getting exactly $k$ is $\frac{\lambda^k}{k!}e^{-k}$. Here, that gives (using Excel for the calculations, since I happen to have it open), probabilities of 1.11145E-08 for $0$, 2.03562E-07 for $1$, and 1.86412E-06 for $2$. The probability of being at most $2$ is their sum 2.07879E-06.
Well, OK, we don't really have all of those digits - but it'll be pretty close. We can confidently say that if the probabilities were accurate, the likelihood of getting at most two successes would be approximately $2\cdot 10^{-6}$. That's definitely a "reject the hypothesis" level.
How many consecutive head flips would it take to reach that level of unlikeliness? The probability of $n$ consecutive flips (with no extra tries) is $2^{-n}$. From the standard approximation $2^{10}\approx 10^3$, we get $10^{-6}\approx 2^{-20}$, and $2^{-19}\approx 2\cdot 10^{-6}$. It would take nineteen consecutive head flips to reach that level of improbability.
I tried a manual version of the binomial approximation as well - it came out about $2.5\%$ different, a total probability of about $2.024\cdot 10^{-6}$. It's technically more accurate, but is that worth switching from a simple exponential $e^{-\lambda}$ to a high power $(1-p)^N$ of something very close to $1$?