I have a question for a challenge that I'm trying to create - having some trouble quantifying the size of the challenge's test bank.
- 20 people are taking a challenge of 9 questions
- the test bank (n) doesn't change. so when a new challenge starts it draws from the same test bank (the questions are not replaced by new ones). within a challenge, however, there are 9 unique questions.
- I need to determine the size of the question bank (n) that yield the acceptable rate of repeats, which is 3 repeats with a probability of 95%.
I previously used combinatorics to solve a problem like this, but I couldn't find out how to correctly integrate the number of people, in this case 20, taking the challenge.
I am going to assume we want less than $5\%$ chance that any pair of the $20$ people share exactly four questions. If that is small, the chance that any pair share five or more is very small, so we can ignore it. It should be clear how to update this analysis to cover that. The chance that a given pair of people share four questions is $\frac {{n \choose 4}{n-4 \choose 4}{n-8 \choose 4}}{{n \choose 8}^2}$ where the numerator comes from choosing the shared questions, the non-shared for the first person, then the non-shared for the second. As there are $\frac 1220\cdot 19=190$ pairs of people, it is a slight overestimate to say there is a $$190\frac {{n \choose 4}{n-4 \choose 4}{n-8 \choose 4}}{{n \choose 8}^2}$$ chance that some pair shares four questions. Alpha says you need 143 questions.