Is this the correct way to conduct a hypothesis test? not sure what to do with $\alpha = 0.01$

Question

The following is an ANOVA table with data testing the relationship between subjective social class and number of science courses taken in college, suggested by Mullen in her article “the not-so-pink ivory tower”. I am want to conduct a hypothesis test and interpret the findings. I am to use $α=0.01$.

Here is the table:

 

What I have:

Ho: There is not a relationship between the variable Subjective Social Class and the variable Numer Of Science Courses Taken In College.

Ha: There is a relationship between the variable Subjective Social Class and the variables Number Of Science Courses Taken In College.

 

And so since $F_{critical} = 20.59$ and $\alpha$ is $0.01$, therefore $F_{critical} > 0.01$ we can reject the null hypothesis giving credence to the alternative hypothesis. Should I say more here. What effect will $p$-value have here on the hypothesis testing and $\alpha$?

Is this right? should I say more?

Thank you

Answer 1

Critical value: The critical value for a test at level $\alpha = 5\%$ where the test statistic has the distribution $\mathsf{F}(\nu_1 = 3, \nu_2 = 2484)$ is the the number $c$ that cuts 5% from the upper tail of $\mathsf{F}(3, 2484).$ In can't be found in most printed tails of F distributions because of the large degrees of freedom in the denominator. However, from software (such as R, used below) we can can find $c = 2.608.$ Because the observed value $F = 20.59 > 2.608,$ we reject $H_0$ at the 5% level of significance.

qf(.95, 3, 2484)
[1] 2.608485

P-value. The P-value is the probability (under $H_0$) of observing a value of $F$ greater than the value 20.59 found in this experiment. From software we see that $P(F > 20.59) = 3.547 \times 10^{-13}.$ Because this is smaller than 0.00005, your software printout shows 0.0000. So the P-value is not exactly $0,$ but very tiny.

1 - pf(20.59, 3, 2484)
## 3.547163e-13

When testing at the significance level $\alpha = 0.05 = 5\%,$ one rejects for P-values smaller than 0.05. So the very small P-value is one way to indicate rejection at levels 5%, 1%, 0.1%, and so on.

The figure below shows the density function (blue curve) of $\mathsf{F}(3, 2484).$ The vertical red broken line indicates the 5% critical value, and the black dotted line (far to the right) indicates the observed value of the F-statistic. Because the density function almost touches $0$ beyond about 5 or 6, the area beneath the density curve to the right of that (and especially to the right of 20.59) is almost $0$.

Notes: (1) In R statistical software pf indicates the CDF of an F-distribution and qf indicates the inverse CDF or quantile function. (2) The 1% critical value is 3.790.

qf(.99, 3, 2484)
[1] 3.789507