Power of a goodness-of-fit test that a die is fair.

Question

We roll a die $n = 450$ times to assess whether it is fair, The null hypothesis $H_0$ (that each of the six faces has probability $1/6$ of showing at any one roll) is to be tested at the 5% level of significance.

Let $X_i$ be the number of times face $i$ shows in $450$ rolls. We use the test statistic

$$Q = \sum_{i=1}^6 (X_i -75)^2/75 \stackrel{\text{aprx}}{\sim} \mathsf{Chisq}(\text{df} = 5),$$ rejecting $H_0$ for $Q \ge 11.0705.$

In fact, the die is biased so that the probabilities of the respective faces are truly $p_1 = (1/6, 1/6, 1/9, 2/9, 1/6, 1/6),$ not $p_0 = (1/6, 1/6, 1/6, 1/6, 1/6, 1/6).$

What is the power of the test against this alternative? That is, what is the probability that $H_0$ is (correctly) rejected?

Note: This is not (yet) a homework or examination question. I will post/accept my solution if I do not see an Answer with a better one.

Answer 1

I tried using SAS to solve this problem, as that is the program we are learning in my course. This seems like a good exercise for me to try. I am pretty inexperienced with comparing multiple probabilities but this is what I tried.

data gene;
input genotype $ count;
datalines;
A 75
B 75
C 50
D 100
E 75
F 75
;
run;
proc freq data=gene;
weight count;
table genotype / testp=(16.67,16.67,16.67,16.67,16.67,16.65);
run;

What my code is doing is testing the hypothesis that each side has a $1\over6$ chance of happening after obtaining $75$ ones, $75$ twos, $50$ threes, $100$ fours, $75$ fives, and $75$ sixes. In this case we would reject the null hypothesis at $\alpha=0.01$.

However, this is not what the question is asking. How could I adjust my program to find the power of the test?

Answer 2

If $p_0$ gives the hypothetical probabilities and $p1$ gives the alternative probabilities against which we seek the power then under that $H_a$ we have $Q \stackrel{aprx}{\sim} \mathsf{Chisq}(\nu=5,\lambda),$ where $\lambda = n\sum_{i=1}^6 (p_{1i}-p_{0i})^2/p_{0i} = 16.67$ is the noncentrality parameter.

Then the power $0.904$ can be found using R statistical software as:

lam=16.66667;  1 - pchisq(11.0705, 5, lam) 
## 0.9038942  # power against specified alternative (noncentral chi-sq)

Here is a simulation based on 100,000 repetitions of the 450 roll experiment:

set.seed(1711)
p1 = c(1/6, 1/6, 1/9, 2/9, 1/6, 1/6)
q = replicate( 10^6, chisq.test(table(sample(1:6,450,rep=T,prob=p1)))$stat ) 
mean(q > qchisq(.96, 5))
## 0.89314   # simulate power

The third decimal of the simulation may not be accurate, but also the noncentral chi-squared distribution is itself an approximation, so it's exact value is not guaranteed either.

The figure below shows the histogram of the simulated alternative distribution, which is approximately $\mathsf{Chisq}(5,16.67)$ [blue curve]. The approximate null density function $\mathsf{Chisq}(5)$ [black curve] and the critical value for a test at level 5% [vertical red line] are also shown.

Note: As suggested in one of my Comments, it may be possible to find a test at the 5% level, sensitive to this particular alternative, that gives higher power.