True or False? And Why? Should I decide based on how low the p-value is? Please help.

Question

If in a Chi-square test the degree of freedom is 2, Test statistic is 5.49, critical value is 3.49. The p-value is computed as 0.032 and confidence level is 95%. The conclusion is that we should reject H0.

Answer 1

No matter the test, the process is always the same - choose the right test, compute a test statistic, get a p-value, make a decision.

You're testing at the $95\%$ level, so we set $\alpha = 0.05$. The rule is if $p < \alpha$ then we reject the null hypothesis. Otherwise, we fail to reject.

For this experiment we have $p = 0.032$. Now this is certainly smaller than $\alpha$. Hence we reject the null hypothesis.

This goes on no matter what test you do, be it $\chi^2$, ANOVA, $F$-test, Kolmogorov-Smirnov, Anderson-Darling, anything.

Answer 2

I suppose your question is how to use the critical value or the P-value to know whether to reject the null hypothesis. The usual terminology for testing is to say that the significance level is 5% (not that the confidence level is 95%, which is used for confidence intervals).

Chi-squared tests are ordinarily right-tailed. That is, large values of the chi-squared statistic indicate poor agreement of the data with the null hypothesis. So if the observed value of the chi-squared statistic is 5.49 and the critical value is 5.99, then you do not reject the null hypothesis at the 5% level. The critical value cuts 5% of the probability from the upper tail of the chi-squared distribution (with the relevant degrees of freedom).

I think you have the wrong critical value and P-value. The critical value for a right-tailed test at the 5% level is 5.99. The P-value corresponding to the observed value 5.49 is 0.642. (You can verify the critical value from a printed table of chi-squared distributions. Typically, some sort of software is required to find P-values.) The computations below are from R statistical software in which qchisq denote a quantile function and pchisq denotes a cumulative distribution function.

qchisq(.95, 2)
## 5.991465
1 - pchisq(5.49, 2)
## 0.0642483

The P-value is the probability under the chi-squared density curve to the right of the observed value 0.064 of the test statistic. If the P-value is smaller than 0.05, one rejects the null hypothesis at the 5% level. So again, using the P-value criterion, you do not reject the null hypothesis.

The figure below shows the density curve for the distribution $\mathsf{Chisq}(df = 2).$ The vertical black line shows the observed value of the test statistic. The vertical broken red line shows the critical value. The area under the curve to the right of the black line is the P-value 0.064; the area to the right of the critical value 5.99 is 0.05.