Two-tailed hypothesis test; Why do we multiply p-value by two?

Question

I understand that in a two-tailed hypothesis test, we must multiply the p-value by two.

i.e. if z=1.95 and it's a one-tailed hypothesis test, our p-value is 0.0256. But, if it's a two-tailed hypothesis test and z=1.95, we must multiply the p-value of 0.0256 by two. Hence, the correct p-value is 0.0512 for the two-tailed hypothesis test.

I can draw it out on the standard normal curve and I understand that we must multiply the p-value by two. But, my question is why we have to multiply by two. What is the conceptual idea behind it?

Answer 1

$2$ because it is two-tailed.

The test is the probability of seeing that value or something more extreme if the null hypothesis is true. $2$ is more extreme than $1.95$; $-3$ is also more extreme.

So you want $\Pr(Z \ge 1.95)+\Pr(Z \le -1.95)$ which, by the symmetry of the normal distribution is equal to $2\times \Pr(Z \ge 1.95)$

Answer 2

What is the conceptual idea behind it?

The idea is that the sign of the z score doesn't matter. What matters is how many sigmas away from the mean the test statistic is, not which direction away it is. Therefore, if we ask, "What is the probability of obtaining z = 1.95?" we mean, in the case of a two-tail test, "What is the probability of obtaining |z| = 1.95?" This is why we must double 0.0256, in calculating the p-value.

It can also be confusing that for a two-tail test, we must halve α, as well as multiplying by 2 in calculating the p-value. But the reasons are similar in each case. In the end, we are comparing the area α, comprising two tails of area α/2, with the area (p-value) under the curve at the tails as defined by |z|.