I am upset with what examples testing "regression to the mean" seem to allude to: People claiming to have "ESP" take a test, and A's score was 2 standard deviations above the mean, whereas B's score was 1 standard deviation below the mean. Should A retake the test? Should B retake the test? (Assume the scores are normally distributed)
The "Correct" answer according to the question
(1) A should retake the test because his score is at the 97.5th percentile. He is expected to score closer to the mean on the retest.
(2) B should not retake the test because her score is at the 16th percentile. He is likely to score closer to the mean on the retest.
My question is that, should those scoring above average then never take a retest??? That seems absurd to me. I think there's a stronger underlying assumption in this problem, namely that people do not really have ESP... correct me if I am wrong.
The point is that among those who score as high as the 97th percentile or as low as the 16th percentile, there are some who are having an unusually high-ESP-score day (in the former case) or an unusually low-ESP-score day (in the latter). Those will regress to the mean. Tomorrow will be someone else's day to have an unusually low-score day or an unusually high-score day, so there will still be just as many in those groups.