Difference between NHST and Fisher approach to decision theory

Question

Is there a difference between the NHST (null hypothesis statistical test) and the Fisher approach to decision theory? I could not find any difference, since it seems to me that they both ignore the alternative hypothesis (which is not ignored for example by the Neyman-Pearson and Bayes approach). But maybe I have problems in finding in the literature a clear description of NHST.

Answer 1

Fisher focused mainly on looking at the probability of outcomes computed under the assumption that the null hypothesis is true. Consider his famous example of a lady tasting tea. She claims tea tastes better if the milk is put into the cup before the tea is poured, than if milk is added after the tea is poured. To test whether she can really tell the difference, she is confronted with eight cups randomly arranged on a tray---four prepared each way. (She is told to pick the four cups that taste best.) Fisher would conclude she can indeed tell the difference if she picks the four milk-first cups, because there is only one chance in ${8 \choose 4} = 70$ of this outcome under the null hypothesis that she cannot distinguish orders of pouring. Fisher developed many kinds of hypothesis tests along these lines, including what we call ANOVA (analysis of variance).

A few years later Neyman and Pearson formulated the framework of acceptance and rejection regions, type I and type II error probabilities, and power. Fisher strongly resisted this framework and apparently especially the idea of power. For the tea-tasting experiment, Neyman & Pearson might have wanted to know the chances that the lady could 'pass' Fisher's test if she had some moderate, but not perfect, ability to distinguish order of pouring. (For example, maybe she can do it for only 80% of the cups she tastes.)

Apparently, it is possible for bright people to read the writings of Fisher on the one hand and Neyman & Pearson on the other hand, and come to remarkably different conclusions about what each is saying and the distinction between their philosophies of hypothesis testing. I have tried to stick to what I view as the clearest and simplest descriptions of the differences. Even so, I will not be surprised if there are lots of Comments trying to 'reinterpret', 'clarify', or 'correct' what I have said. (Perhaps some from people who have never read a word of the writings of Fisher or of Neyman & Pearson.)

What is totally obvious is that there was a long and bitter battle of Fisher vs. Neyman & Pearson over fundamental ideas of hypothesis testing. Fortunately, most texts attempt to explain the formulation and testing of hypotheses is a way that makes sense to students without feeling the need to go into past controversies.

Note (probably unrelated to your specific question): There is also a Bayesian approach to hypothesis testing. It is somewhat controversial but not in a way directly related to the Fisher vs Neyman-Pearson debate. Very roughly, a Bayesian might require a higher level of proof if the lady claims she can tell the difference in order of pouring by closing her eyes and (without smelling or tasting) sensing 'vibrations in the cup's aura' that she claims differ by order of pouring. And a lower level of proof if she only claims that Darjeeling and Earl Grey teas differ noticeably in taste. Bayesians start out with a 'prior probability' that the lady has the ability she claims, and put that together with the observational data to get a 'posterior probability', which they use to make an inference.