The following question is from a high school statistics exam with the answer posted below the question. My question is if you use the entire strata instead of a simple random sample, why is this not a legitimate way to calculate statistical comparisons?
Question: Any statistical test that is used to determine whether the mean student to teacher ratio is the same for the top 10 performing schools as it is for the bottom 10 schools schools would be inappropriate. Explain why in a few sentences.
Answer: Essentially correct (E) if the response states that the data are not samples from some larger population OR that they are not random samples but instead are those with the highest and lowest proportions of students meeting a standard, and therefore inference is not appropriate. The response must not include any other reason (such as small sample sizes or the shape of the distribution).
Comparing all top ten districts with all bottom all ten may provide useful information. (And the numbers might serve as a leading 'nugget' of information for an newspaper story or journal article.) The issue here is that you have surveyed the entire relevant (but small) populations.
Not all worthwhile information requires formal statistical tests. No statistical test is appropriate here because none is needed. Either the two groups have "impressively different" student-teacher ratios or they don't. But it's up to the person seeing these numbers to decide what "impressive" means.
By contrast, if you had a random sample from a large population of school districts with one dependent variable measuring 'performance' and one explanatory variable 'student-teacher' ratio, then you might see if there is a significantly positive Pearson correlation, and if so do a regression analysis to explore the linear relationship. The correlation and regression procedures would involve statistical tests.