Confusion regarding the p- value of a test

Question

I have a confusion regarding the p-value of a statistical test. Suppose $\tau$ is a normally distributed rv. We have for a one tailed test: p value=$P(\tau > \tau_{0})$ (for right tailed test), p value=$P(\tau < \tau_{0})$ (for left tailed test) $\tau_{0}$ is the observed value of the test statistic. Now, I am having doubts how to interpret the p value of a both tailed test which is given by $2 P(\tau > |\tau_{0}|)$ Also, had the distribution been a positively skewed one with $\chi$ being positive (as in chi-squared distribution) how can we interpret the p-value for a both tailed test?

Answer 1

Here is a discussion in terms of two-sided confidence intervals, rather than two-sided tests, that illustrates inference for exponential data.

First, if the exponential mean is $\mu,$ rate $\lambda = 1/\mu,$ then the distribution of the sample mean $\bar X$ is $\mathsf{Gamma}(n, n/\mu),$ a gamma distribution with shape parameter $n$ and rate parameter $n/\mu$. Thus $E(\bar X) = \mu$ and $SD(\bar X) = \mu/\sqrt{n}.$ Also, $\bar X/\mu \sim \mathsf{Gamma}(n.n).$

This can be proved by using moment generating functions, and is illustrated for the case $\mu = 1, n = 16$ by the brief simulation in R statistical software below. [In R, rexp samples from an exponential distribution, dgamma is a gamma density function (second parameter, rate), qgamma is the inverse of a gamma CDF, and a is a vector of a million sample means.]

set.seed(318)
a = replicate(10^6, mean(rexp(16)))
mean(a);  sd(a)
[1] 0.9999153  # approx 1
[1] 0.2499526  # approx 1/4 = 1/sqrt(16)
hist(a, prob=T, col="skyblue2", main="GAMMA(16,16)")
  curve(dgamma(x,16,16), add=T, col="blue", lwd=2)
  abline(v=qgamma(c(.025,.975),16,16), col="red", lwd=3, lty="dashed")

Now, consider a particular random sample of size $n = 16$ from $\mathsf{Exp}(rate=1/10).$

set.seed(325);  x = rexp(16, .1)
mean(x);  sd(x)
[1] 9.888107
[1] 8.290735
stripchart(x, pch="|")

Of course, because we simulated the data we know that $\mu = 10.$ But let's use the data to make a 95% confidence interval (CI) for $\mu$ and see what happens. We have

$$P\left(L < \frac{\bar X}{\mu} < U\right) = P\left(\frac 1 U < \frac{\mu}{\bar X} < \frac 1 L\right) = P\left(\frac{\bar X}{U} < \mu < \frac{\bar X}{L}\right) = 0.95,$$

where $L$ and $U$ cut 2.5% of the probability from the lower and upper tails, respectively, of $\mathsf{Gamma}(16,16)$ [see the vertical dashed lines in the first figure above.] Thus a 95% CI is $(\bar X/U, \bar X/L) = (6.395, 17.299),$ which has length 10.905 and covers the true value $\mu = 10.$ This CI is sometimes called 'probability-symmetric' because it was formed by taking equal probabilities from the two tails of the appropriate distribution.

CI = mean(x)/qgamma(c(.975, .025), 16, 16);  CI;  diff(CI) 
[1]  6.394839 17.299409
[1] 10.90457

However, it is possible to get a shorter 95% CI by cutting different probabilities from the two tails: $(5.916, 16.259)$ of length 10.343. With some fussing (or a simple computer search) one could find the interval that is, for practical purposes, shortest.

CI.2 = mean(x)/qgamma(c(.99,.04), 16, 16);  CI.2;  diff(CI.2)
[1]  5.915955 16.259290
[1] 10.34334

In a similar way, one usually has to use computational methods to find the best test of $H_0: \mu = \mu_0$ against $H_1: \mu \ne \mu_0.$ Or to find its true P-value for a particular set of data.

Note: If one (incorrectly) assumes data are normal and uses a 95% CI from Student's t distribution, the result is $(5.470, 14.306).$ Such intervals (using both the sample mean and SD along with the incorrect assumption) are often shorter than the ones made from a gamma distribution, but they do not truly have the 'advertised' 95% coverage probability. [A Shapiro-Wilk test of normality for these 16 observations convincingly rejects the null hypothesis with P-value 0.0157.]