Randomized and nonrandomized statistical tests

57 Views Asked by At

I've been reading recently about randomized and non-randomized statistical tests and their differences. If I understand the definition correctly, a nonrandomized test for all values of the test statistics rejects the null hypothesis with probability $1$ or $0$. For a randomized test, for certain values of the test statistics, the probability of rejecting $H_0$ is different from $0$ and different from $1$.

An example of such a randomized test is the exact test for the probability of success from the binomial distribution (binom test, https://en.wikipedia.org/wiki/Binomial_test). If I understand correctly, the asymptotic test written on Wikipedia is already non-randomized.

I am also considering the definition of the test power function. The version I knew before was just 1 minus type II error. Now, I read that this is a function of the tested $\theta$ parameter and is defined as the expected value $\phi(X)$, where $\phi$ is our statistical test, i.e. a function describing the probability of rejection of $H_0$, which is a function of the random variable $X$ that we are examining. In the case of a non-randomized test, $\phi(X)$ takes only the values $0$ or $1$, so the power function of the test is equal to the type I error when $\theta = \theta_0$ and 1 - type II error when $\theta\neq\theta_0$. If I understand correctly, for a randomized test it is not necessary that the power of the test = type I error or 1-type II error, because with non-zero probability the function $\phi(X)$ takes values other than 0 and 1.

Do I understand all this correctly?

Therefore, if in the simulation task, I should compare the powers of the binom test and the asymptotic test for different values of $\theta$, it is probably not enough in the case of binom test to examine the type 1-type II error for various alternatives? i.e. simulate data not fulfilling $H_0$ and checking how many times $H_0$ was rejected in favor of $H_1$. This will not be the power of the test in the case of randomized test, if I correctly understand the exact definition.