Sampling $1000$ birds from a population which has $10$ types of birds the expected outcome is $100$ birds of each type (this is for the sake of simplicity; general case is each distributed with Probability $P_i$ where $i = 1..10$).
Now I have written a program to randomly sample $1000$ numbers (read birds) with each type with equal probability (or probability $P_i$ where $i = 1..10$ in general case)
Let say expected outcome is $E$ and during any run of program the observed outcome is $S$. Now to write a unit test case for this program, I have performed Chi-Squared test to compared $S$ to $E$ and asserted that p-value of Chi-statistic is greater than 0.05. I have asserted this $n$ (say $100$) times in a loop.
It would not be correct to expect each of $n$ p-values to be $>0.05$. So is it a good idea to assert that "average of" $n$ p-values to be $>0.05$.
More generally, what is the right approach in designing tests here?
Is Kolmogorov-Smirnov two-sample test a better test here?