Is it possible to eliminate completly the type $1$ error risk?

672 Views Asked by At

I was reading here about type $1$ and type $2$ errors and I have a question.

Since "No hypothesis test is 100% certain. Because the test is based on probabilities, there is always a chance of making an incorrect conclusion." Does this mean that is impossible to lower the risk so that the risk to make a type $1$ eror is actually $0$?

Or can there be a way to obtain the risk of making a type $1$ error $0$?

What about the type $2$ error? Is it possible to make it's risk $0$?

I am looking forward for a more detailed/rigurous answer for this and not necesary depending on a specific problem (A more general case).

3

There are 3 best solutions below

0
On BEST ANSWER

Usually there is no way for the risk of making a Type I error to be $0$. Suppose that you have a test statistic for your null hypothesis and you set a threshold, saying "If the test statistic exceeds this threshold then I will reject the null hypothesis". If there is any possibility that the test statistic will exceed this threshold under the null hypothesis then the Type I error rate will be greater than $0$.

For example, suppose that $X_1, \dots, X_n$ are known to be independent and identically distributed random variables with each $X_i \sim N(\mu, 1)$ (mean $\mu$, variance $1$), where the mean is unknown. Let the null hypothesis be that $\mu = 0$ and let the alternative be that $\mu > 0$. We consider the test statistic $\bar{X} = \frac{1}{n} \sum_{i = 1}^n X_i$. Under the null hypothesis we have $\bar{X} \sim N(0, 1/n)$ (mean $0$, variance $1/n$). If you say, "I will reject the null hypothesis if $\bar{X}$ exceeds threshold $T$" then no matter how large $T$ is (as long as it is finite), we have $P(\bar{X} > T) > 0$ and this is our probability of Type I error. If $T = \infty$ then you are in the trivial case where your Type I error is $0$ because it is impossible for you to reject the null hypothesis under any circumstances.

Occasionally you can have cases where you can certainly reject the null hypothesis. E.g. if your null hypothesis is that $T$ is an upper bound for observations $X_1, \dots, X_n$ and your alternative is that the true upper bound is higher than $T$, then as soon as you observe some $X_i > T$ then you can reject the null hypothesis with no chance of Type I error, since it would be impossible to see $X_i > T$ under the null hypothesis.

The same argument applies for Type II error: if there is any possibility that the data will look like they were generated under the assumptions of the null hypothesis when they are actually generated from the alternative, then you have a non-zero probability of Type II error.

2
On

With these triangular distributions, if you set your criterion as shown by the red dashed line, you will never make a mistake by categorizing a member of the blue category as a member of the tan category.

two triangular distributions

2
On

If you want absolutely to avoid type I error in a pregnancy test, always say "you are not pregnant"

enter image description here

Edit: you will find more about the picture and type I and II errors here:
http://www.sethspielman.org/courses/geog5023/r_examples/Type_I_and_II_Errors.html