Change in Overall Ratio is Opposite of Change in Groupwise Ratios

Question

The table below displays group-wise success rates of Asian, black and white students as well as their totals in 2020 and 2021.

How is it possible that all three groups became less successful from 2021 to 2021 but the total success rate went up?

Is there a mathematical formulation of how and when this may be possible?

            2020                2021                
    Successful  Total   Successful  Total   Success Rate 2020   Success Rate 2021   Change
Asian   270      354    343          487        76.3%               70.4%           -5.8%
Black   465      710    449          706        65.5%               63.6%           -1.9%
White   2320    7122    490         1518        32.6%               32.3%           -0.3%
Total   3055    8186    1282        2711        37.3%               47.3%            10.0%

Answer 1

The average success rate increases if the share of a group with an $\mathrm{\color{green}{above-average}}$ success rate increases or the share of a group with a $\mathrm{\color{red}{below-average}}$ success rate decreases.

$$\begin{array}{cc|cc} && \text{SR} & \text{share} \\ \hline 270 & 354 & \color{green}{.76} & .04 \\ 465 & 710 & \color{green}{.65} & .09 \\ 2320 & 7122 & \color{red}{.33} & .87 \\ \hline 3055 & 8186 & .37 \end{array} \qquad \begin{array}{cc|ccl} && \text{SR} & \text{share} \\ \hline 343 & 487 & .70 & .18 & \uparrow \\ 449 & 706 & .64 & .26 & \uparrow \\ 490 & 1518 & .32 & .56 & \downarrow \\ \hline 1282 & 2711 & .47 \end{array}$$

This can be seen geometrically: If we plot the number of successes against the number of attempts each group corresponds to a vector with its "slope" being the success rate.

$\hspace{3.5cm}$

In this example the average success rate (the slope of their sum) increases if $u$ gets shorter or $w$ gets longer, but is independent of the length of $v$ since they have the same slope.

In general, the change of the average success rate depends on the change of the group-wise success rates and "weights" with respect to the initial average success rate.

Answer 2

In this type of problems, the overall ratio is the weighted average of the group ratios, where the weight is given by the relative sizes of the groups. It us useful to consider a simple scenario with only two groups, as the main concept is the same for the cases with a higher number of groups.

With two groups, if $s_1$ and $w_1$ are the success rate and the weight in group 1, and $s_2$ and $w_2$ are the corresponding values in group 2, we have

$$S= s_1 w_1+s_2w_2$$

where $S$ is intermediate between $s_1$ and $s_2$. For example, let us consider a total population of $100$ subjects, of which $60$ are in group 1 and $40$ are in group 2. Assume that group 1 has a success rate of $s_1=45/60=0.75$ and group 2 has a success rate of $s_2=4/40=0.1$. The first ratio has weight $w_1=60/100=0.6$ and the second ratio has weight $w_2=40/100=0.4$. The overall success rate is then

$$S= 0.75 \cdot 0.6 +0.1 \cdot 0.4=0.49$$

Note that the position of $S$ within the range between $s_1$ and $s_2$ is only determined by the weights. For given $s_1$ and $s_2$, $S$ will progressively approach the value of $s_1$ as group 1 becomes larger than group 2, and vice versa.

Now let us hypothesize that, in a second evaluation, both $s_1$ and $s_2$ decrease. This is a situation similar to that reported in the OP. Compared to the first evaluation, if now $w_1$ and $w_2$ are kept constant, this clearly leads to a decrease in $S$ as well. However, if we allow $w_1$ and $w_2$ to change, $S$ is no longer necessarily reduced. In fact, let assume, by simplicity, that in both evaluations $s_1>s_2$. In the second evaluation, if group 1 increases its weight, the overall $S$ is moved nearer to $s_1$, and this effect can overcome that of reduced group success rates.

Just an example: let us take the groups above. Let us hypothesize a second evaluation where the total sample size is again $100$, and both $s_1=0.7$ and $s_2=0.05$ are decreased. If the weights are kept unchanged, we get

$$S=0.7 \cdot 0.6 + 0.05\cdot 0.4= 0.42$$

and $S$ is reduced, as expected.

However, if group 1 has increased its weight in this second evaluation (for example, it includes 80 subjects out of $100$), we have $w_1=0.8$ and $w_2=0.2$. The overall success rate is then

$$S= 0.7 \cdot 0.8 +0.05 \cdot 0.2=0.57$$

So $S$ is increased in comparison with the first evaluation, because the effect due to the larger weight of group 1 exceeds that of reduced group success rates.

Extending these considerations to the cases with $>2$ groups: in general, $S$ can be increased, despite a reduction in group success rates, when the relative size of the group(s) with highest rates are sufficiently increased to overcome the effect of reduced success rates. Actually, this is what happened in the calculations shown in the OP, since the groups with highest rates (Asian and Black) considerably increased their weight from 2020 to 2021. The weight of the Asian group changed from $354/8186\approx0.043$ to $487/2711\approx0.18$. The weight of the Black group changed from $710/8186\approx0.087$ to $706/2711\approx0.26$.