Type I Error and Type II Error

Question

Rigourously speaking, can I say that the type I error is the probability that I reject the null, when the null is true? I've seen some say that is incorrect to say, since type I error is defined to be the abovementioned event , not a probability. The probability, per se, of type I error is named the significance level of the hypothesis test. Is that the correct approach?

Answer 1

Type I error is the event, not the probability of the event.

The significance level is the probability of that event, provided there is only one probability distribution consistent with the null hypothesis.

But often there is a compound null hypothesis. For example, you may have $X\sim N(\mu,\sigma^2)$ and the null hypothesis says $\mu\le 0$. In that case, the probability of rejecting the null hypothesis, given that it is true, depends on $\mu$. But for one value of $\mu$ (which in this case is $0$ if you're using the uniformly most powerful test) the probability of Type I error is larger than for any other value of $\mu$. Then the significance level is that largest value.

Answer 2

I think your initial sentence is fine if you add the word "rate": "the type I error rate is the probability that I reject the null" As you mentioned, a Type I error is an event on which you wrongly reject H0. Also as you can read on wikipedia, talking about significance level can have multiple interpretations, so I would stick to "type I error rate".