Check if set of dice is fair

Question

I had a statistics question that I was curious about. Hypothetically, a company found a new way to manufacture dice. If I wanted to check if the dice are fair with 1/6 probability for each side, how would I go about doing that. There would be N dice that are rolled n times which is N * n total dice throws.

How would you go about statistically verifying that the process manufactures fair dice? Is there a way to measure the uncertainty of the dice being fair?

Something that I thought could work are using the chi-squared test and goodness of fit but this could get tedious for a large amount of dice and dice throws. Are there other statistical tests that would be better suited to address this question?

Answer 1

You're not asking the right question. We know for sure it won't manufacture fair dice. Any physical object has probability $0$ of being exactly fair. The probabilities of the different outcomes might be very close to $1/6$, but they won't be exactly $1/6$. And if by some miracle they were exactly $1/6$ there would be no way of verifying that fact. What you might do is try to estimate the amount of unfairness.

Answer 2

Yes, you could use a chi-squared test, to check whether the die is nearly fair.

Simulate 600 dice rolls. Suppose you get the following tally from 600 rolls of one of the new style of dice. [Dice rolls simulated using R.]

set.seed(929)
x = sample(1:6, 600, rep=T)
table(x)
x
  1   2   3   4   5   6 
108 105 106  99  88  94 

f = c(100, 106, 106, 99, 88, 94)

Chi-squared test in R.

chisq.test(f)

       Chi-squared test for given probabilities

data:  f
X-squared = 3.06, df = 5, p-value = 0.6907

If you have not done a chi-squared test using the formula above, you should do so to verify the above result. Such a simple type of computation might show up on a test

[The given probabilities are assumed to be all equal, unless some other null distribution is provided. In the output, the chi-squared statistic is denoted X-squared, not $Q,$ as shown in the displayed equation below.]

Under the null hypothesis that all six faces are equally likely, the six expected counts are $E_i = 100,$ and the chi-squared statistic $$Q = \sum_{i=1}^6 \frac{(f_i - E_i)^2}{E_i} = 3.06,$$ which has approximately the distribution $\mathsf{Chisq}(\nu = 5).$

Die 'passes' as fair. The null hypothesis is would be rejected at the 5% level of significance if $Q > 11.0705.$ So the null hypothesis is not rejected: based on data from our 600 rolls, we have no reason to suspect that the die is unfair.

qchisq(.95, 5)
[1] 11.0705

The figure below shows the density function of the distribution $\mathsf{Chisq}(5).$ The critical value $11.0705$ (also obtainable from printed tables of chi-squared distributions), is shown as a vertical dotted line. The observed value of the chi-squared test statistic $Q = 3.06$ is shown by a solid vertical line. The P-value of the test $0.6907$ (much above 5%) is the area under the density curve to the right of this solid line.

curve(dchisq(x,5), 0, 15, lwd=2, ylab="PDF", xlab="Chi-sq",
      main="Density of CHISQ(5)")
abline(h=0, col="green2");  abline(v=0, col="green2")
abline(v=11.07, col="red", lty="dotted")
abline(v=3.06, lwd=2)

Are 600 rolls enough? The question arises whether 600 rolls of the die are sufficient to reject the null hypothesis---if the die were unfair. It is possible to do a 'power and sample' size computation to show that 600 rolls are sufficient. Perhaps more conveniently, we can simulate 600 rolls of a die that is slightly unfair, with probabilities $(2/18, 3/18, 3/18, 3/18, 3/18, 4/18),$ slightly favoring 6s over 1s.

We show just the chi-squared statistic $23.42,$ which exceeds the 'critical value' 11.0705, indicating that the die is unfair.

set.seed(2001)
x = sample(1:6, 600, rep=T, pr=c(2,3,3,3,3,4)/18)
chisq.test(tabulate(x))$stat
X-squared 
    23.42 

Several subsequent tests with unfair dice, also reject the null hypothesis, so it seems 600 is plenty of rolls of the die---if we are not too fussy. However 60 would not have been enough.

set.seed(2020)
x = sample(1:6, 60, rep=T, pr=c(2,3,3,3,3,4)/36)
chisq.test(tabulate(x))$stat
X-squared 
      4.6 

Addendum: Illustrating @RobertIsrael's Comment. If a die with the slight bias shown below is rolled 6000 times, it will be detected as biased only about 1/3 of the time, according to a simulation of 100,000 tests with 6000 rolls.

bias = c(19,20,20,20,20,21)/120
set.seed(1234)
q = replicate(10^5,chisq.test(tabulate(
              sample(1:6,6000,rep=T,p=bias)))$stat)
mean(q >= 11.07)
[1] 0.36061

A casino might make some crooked money over the long run using such biased dice in specific games, but it seems unlikely that an individual customer would detect the dice are biased. [That's the reason most juristictions that allow casinos have gaming commissions to regulate them.]