Correct statistical method to test for a change in percentile (not mean)?

82 Views Asked by At

I want to test the impact of a process change on an operational metric, which is a continuous variable. I have two data sets, pre-test and post-test. Both data sets represent the entire population of events that occurred during the specified time periods, and both have a population size of > 5,000. I want to know if there was a positive or negative change following the intervention and whether that change is statistically significant.

My intuition is to apply a two-tailed z-test, however this particular metric is reported using its 90th percentile rather than its mean. A z-test for proportion doesn't seem to fit either. Essentially, I want to know if a change in the 90th percentile was statistically significant.

1

There are 1 best solutions below

1
On

Population 1 (size 5000):

set.seed(2020)
x1 = rnorm(5000, 100, 15)
quantile(x1, .90)
     90% 
118.9256 

Population 2 (size 6000):

x2 = rnorm(6000, 110, 15)
quantile(x2, .90)
     0% 
129.5449 

The 90th percentile has increased from 118.9 to 129.5. There is no doubt about it.

[Sampling and quantile computation in R.]