Should the choice between two sided or one sided test be based on sample's data?

Question

"Suppose the assets price follow a normal distribution with variance 9. The sample mean of 10 assets is equal to 11.15. The manager of the investment fund says that the population mean is 12.5. Using the sample mean, find the p-value. Would you say that the manager is correct?" So in this question I used a two sided test to calculate the p-value. However, my teacher said that it was supposed to use a one sided test, because the sample mean was less than the population mean. I tried to argue with him, but he called my idea stupid. I know it has something to do with data snooping. If it really is a case of data dredging, how can I prove him wrong?

Answer 1

I have no advice for winning arguments with instructors. But I can comment on the choice between one- and two-sided alternatives.

The fundamental guideline is this: Ideally, in a real application, the choice of alternative should be made before seeing the data. Here are some scenarios to consider:

(a) If you have a drug that is supposed to decrease blood pressure, but it has not been tested on human subjects before, you can't be sure that subjects' blood pressure will drop after taking the drug. Sometimes drugs have been known to have an effect opposite to the one intended. So you should have a two-sided alternative.

(b) If you are debating whether a certain amount of alcohol will change a drivers' reaction times, then it seems safe to say that the effect will be to increase (slow) reaction time. There have been enough tests of the effect of alcohol on reaction time that the only issue is whether the dose we have in mind is large enough to have a detectably bad effect.

Here are some considerations in deciding the type of alternative:

(a) If you have a one-sided alternative and the data turn out to be strongly in the opposite direction to what you expected, then the only honest conclusion is that that you can't reject the null hypothesis in favor of the alternative you had in mind.

(b) Often researchers like to use one sided alternatives. If the experiment goes in the direction they intended the P-value will typically be half what it would have been for a two-sided alternative, so it is easier to reject the null hypothesis and claim the experiment was a success.

(c) In experiments that might lead to US government approval for a drug or process, it is required to say before data are collected whether the alternative is one or two-sided, and it is a violation of regulations or law to change the kind of alternative after seeing the data.

However, it is not usually possible to give all the context and details in a briefly worded textbook problem. So problems often use 'code words' to send a message whether a one or two-sided alternative is intended. Words like "increase" and "decrease" are used as cues for a one-sided alternative, and phrases "made a change", "had a significant effect", or "is the claim correct" as cues for a two-sided alternative.

---

The problem you mention may a more interesting illustration of computing a P-value if you use a one-sided alternative. It might be uninteresting to say that the fund manager had understated the value of his fund. In your problem, you have $n = 10, \sigma = 9, \bar X = 11.15,$ and $\mu_0 = 12.5.$ So the test statistic is $Z = \frac{11.15 - 12.5}{9/\sqrt{10}} = -0.4743.$

The interesting issue seems that the manager may have overstated the value of the fund, and we wonder if $\bar X$ is enough smaller than $\mu_0$ to show a significant overstatement.

Admittedly, this is a scenario based on seeing the data. But as an exercise in statistical computation (if not statistical ethics), I can see the point in wanting to use $H_0: \mu = 12.5$ and $H_a: \mu < 12.5.$ Then the P-value is $\Phi(-0.4743) \approx 0.32$ and you won't reject at the 5% level or any other reasonable level. If the alternative were $H_a: \mu \ne 12.5,$ then the P-value would be about $0.64$ and you're even farther away from rejection.

The bottom line is that I think you have a very good grasp of the practical issues involved in choosing the type of alternative. Whether it is a good choice to press your valid point of view with the instructor is up to you.

Answer 2

A two sided test is recommended if you have a null hypothesis and an alternate hypothesis defined such as below:

$H_0: \mu = 12.5$

$H_1: \mu \ne 12.5$

A one sided test is recommended if you have a null hypothesis and an alternate hypothesis defined such as below:

$H_0: \mu = 12.5$

$H_1: \mu \lt 12.5$

If the manager claims that the population mean is less than 12.5 and that you would want to test if he is correct, then your prof is right in suggesting a one tailed test.

Since in the wording, the manager has claimed that the population mean is 12.5 and that you want to test if he is correct, you are right in suggesting a two tailed test.

There could be some problem in interpreting the wording of the question. Before I say that your prof is wrong, make sure the context and the wording of the problem is correct with him.