Test of hypothesis on the difference between population groups- deciding on the method to use

Question

The problem is given as: In order to investigate the opinion of three regions on the passing of a new policy, random samples were obtained from the regions. The table below shows the results. Use test of hypothesis (at 5% significance level) to conclude if the approval rates are different among the regions.

And I'm trying to decide on what method to use for testing the difference between the regions. I thought about using one of the post hoc comparisons, something like the LSD method, but the test statistic here isn't the population mean. Would one use the test on homogeneity here? It was the only thing that came to my mind for hypothesis testing on proportions, but I'm not entirely sure.

Answer 1

Test of homogeneity. Test the null hypothesis is that Yes/No proportions are constant across the three Regions. This is a chi-squared test with degrees of freedom $(3-1)(2-1) = 2.$ I will show the method with some explanations along the way.

Analysis in R, begins by making a table of counts, such as the one below for data similar to yours.

TAB = matrix(c(198,202, 149,210, 133,217), byrow=T, nrow=3)
TAB
     [,1] [,2]
[1,]  198  202
[2,]  149  210
[3,]  133  217

chisq.out = chisq.test(TAB)
chisq.out

        Pearson's Chi-squared test

data:  TAB
X-squared = 10.74, df = 2, p-value = 0.004655

The null hypothesis of homogeneity is rejected with P-value 0.005. (For your exact data, P-value is a little smaller.)

Now look at Pearson residuals to see where inconsistencies may lie.

• The chi-squared statistic is a sum of six components $q_{ij}=(X_{ij} - E_{ij})^2/E_{ij},$ for $i=1,2,3$ rows; $j=1,2$ columns, where $X_{ij}$ are observed counts. Expected counts $E_{ij}$ are computed from row and column totals assuming homogeneity.

• Pearson residuals $r_{ij}$ are square roots of the $q_{ij},$ except that they keep the signs of the $q_{ij}.$ (They're called 'signed square roots'.)

The residuals largest in absolute value are a clue to which components are most responsible for having a chi-squared statistic large enough to reject.

Here are observed counts, expected counts, and Pearson residuals.

chisq.out$obs
     [,1] [,2]
[1,]  198  202
[2,]  149  210
[3,]  133  217
chisq.out$exp
          [,1]     [,2]
 [1,] 173.1289 226.8711
 [2,] 155.3832 203.6168
 [3,] 151.4878 198.5122
chisq.out$res
           [,1]       [,2]
[1,]  1.8902070 -1.6512189
[2,] -0.5120806  0.4473357
[3,] -1.5020936  1.3121766

• The residual with the largest absolute value is 1.89. It calls attention to the observed count 198 for Yes in Region 1 that is much larger than the corresponding expected count 173.1.
• Also, the residual -1.50 calls attention to the observed count 133 for Yes in Region 3 that is much smaller than the corresponding expected count 151.5. So it seems the Regions 1 and 3 disagree.

You might do a post hoc test of proportions in favor to compare Regions 1 and 3. Notice the use of row totals from the original data table in your Question.

prop.test(c(198,133), c(400,350))

        2-sample test for equality of proportions 
        with continuity correction

data:  c(198, 133) out of c(400, 350)
X-squared = 9.5515, df = 1, p-value = 0.001998
alternative hypothesis: two.sided
95 percent confidence interval:
 0.04170612 0.18829388
sample estimates:
prop 1 prop 2 
 0.495  0.380 

This test rejects equality of proportions in the two regions with P-value 0.002. In order to avoid 'false discovery' by making repeated tests on the same data, you should use some method for deciding on a level of significance below the usual 5% level. (The Bonferroni method is one possibility.) But this P-value 0.002 is sufficiently small to be taken seriously.

You may want to make other comparisons among regions, but my guess is that the difference between Regions 1 and 3 may be the most important finding.