ANOVA F-test results in p-value being 1

Question

In one of the statistics exercises I have tried to solve, I have to show that the means of the observations in four groups aren't equal.

I read that a t-test can only be used to compare means for two groups and that a solution would be to make an ANOVA which includes an F-test.

The result of my F-test ended up being approximately 62. Since this is far away from 1, it indicates there is a difference in the means. However, I would like to know whether 62 sounds legit? (I know you haven't seen the exercise).

Furthermore, how can I calculate the p-value by myself? I tried to calculate the p-value in Excel but that results in value 1 and I don't know if that is correct. Of course, 1 is much bigger than my statistical significance 0,05.

Answer 1

If the F-statistic is $62$ then the p-value should be nearly $0.$ An F-statistic equal to $62$ is perfectly plausible if there are clear differences among the groups. That the F-statistic should be near $1$ is something that holds if it actually has an F-distribution, and that is something that should hold if the null hypothesis is true, but not if it is false.

Answer 2

I suppose you must have $r = 15$ replications in each of your $g = 4$ groups. According to the null hypothesis (all four group population means equal), the F-statistic has distribution $\mathsf{F}(3, 56).$ The critical values for testing at the 5% and 1% levels are 2.796 and 4.152, respectively, as computed in R statistical software. (The idea is to cut 5% or 1% of the area from the upper tail of the distribution $\mathsf{F}(3, 56).$ In R, qf is the inverse CDF or quantile function of F; maybe you can figure out how to get similar results from Excel.) So with observed F of 62.911, you are far above the critical value for any reasonable significance level.

qf(.95, 3, 56);  qf(.99, 3, 56)
## 2.769431    # 5% crit val
## 4.151941    # 1%

If the observed $F$ is 62.911, then the P-value is the probability $P(F > 62.911) $ $= 1 - P(F \le 62.911) \approx 0,$ where $F \sim \mathsf{F}(3,56).$ The P-value is the probability of a more extreme result than what you observed; here, 'extreme' means farther out into the right tail. From R, we have the following.

1 - pf(62.911, 3, 56)  # 'pf' is the CDF
##  4.440892e-16

If there are relatively large difference among the four sample means of your groups, then there may be large differences among the corresponding population means, and it is hardly surprising to get an observed F-value as large as yours.

Below is a graph of the PDF of $\mathsf{F}(3,56),$ with vertical red lines at the critical values (1% dashed). You can see that there is almost no probability under the PDF to the right of 63.