I'm trying to investigate whether mean concentrations of a particular hormone significantly differ in two groups. However, one group has 1000 samples, and another has about 50.
I have found one approach is reducing the majority group to match the smaller group, i.e. getting rid of 950 samples randomly. However, this approach seems wasteful because we are now getting rid of 95% of the data in the first group.
Is it possible to divide up the large group into 20 groups with N = 50, and do a t-test for all 20 smaller sized groups at equal sizes? If so, how do I report back one p-value? Another approach I wanted to try is repeated random under sampling ( similar to the first approach but the groups are created randomly) to match the smaller group, but run into the same issue where I then have as many p-values as repeated trials.
I know there are different test that can be run such as Mann Whitney and Welch's, but I really wanted to see whether a t-test will suffice doing multiple comparisons?
Thanks
The Welch test is a $t$-test, because the test statistic is, under certain assumptions, approximately $t$-distributed. In fact, if the within-group variances are roughly equal, there is nothing stopping you from performing the usual $2$-sample $t$-test. The larger group would have an asymptotically normal sample mean except under severe violations of the Central Limit Theorem. The small group, while small relative to the larger group, is still $n_2 = 50$ observations at which point the critical value for the test is very close to a standard normal critical value. In fact, the Satterthwaite approximation for the degrees of freedom for a Welch $t$-test would be very close to $n_2 - 1 = 49$, except if the variance of the first group is enormous even with the large data size.
The takeaway here is that you should not be concerned about the difference in sample sizes. Of more concern would be deviations from normality or other violations of the test assumptions. I would not attempt to do the kind of procedure you are proposing; this would most likely lose an enormous amount of statistical power, or inflate the Type I error due to multiple comparisons. There simply is no need to go with such an elaborate testing procedure.