Suppose a random sample of size 10 is taken from the random variable X which has the normal distribution with unknown mean $\mu$ and variance 4. You are requested to test the hypothesis $H_0:\mu = 0$ against $H_1:\mu \ne 0$ with a $5\%$ significance level. Derive the power function for this hypothesis test, and evaluate it when $\mu = −1$, $\mu = 0$, and $\mu=1$ .
The above is my question. I understand this will be a Z test, which would give the following: $$Z=\frac{µ-0}{2/\sqrt{10}}$$ For a $5\%$ significance level it would be $1.96$, but I'm just not sure where to go from here. Plugging values into $\mu$ gives $-1.5811$, $0$ and $1.5811$. How do I evaluate these numbers? Many thanks
The distribution of individual observations: $$X_i \sim \operatorname{Normal}(\mu, \sigma^2 = 4).$$
The sample size: $n = 10$.
The hypothesized mean: $\mu_0 = 0$.
The hypothesis:
$$ H_0 : \mu = \mu_0 \quad \text{vs.} \quad H_1 : \mu \ne \mu_0.$$
The sampling distribution of the sample mean:
$$\bar X = \frac{1}{n} \sum_{i=1}^n X_i \sim \operatorname{Normal}(\mu, \sigma^2/n).$$
The test statistic under the assumption that the null hypothesis is true:
$$Z \mid H_0 = \frac{\bar X - \mu_0}{\sigma/\sqrt{n}} \sim \operatorname{Normal}(0,1).$$
The significance level: $\alpha = 0.05$.
The critical value that the absolute value of the test statistic must exceed in order to reject $H_0$: $\Pr[|Z| > z^*] = 0.05$ implies $z^* \approx 1.96$.
The probability of rejecting $H_0$ when $H_1$ is true: $$\Pr[|Z| > z^* \mid \mu = \mu_1]$$ for some alternative mean $\mu_1 \ne 0$. Under the assumption $\mu = \mu_1$, $Z$ is no longer a standard normal variable. Its variance is still $1$ but its mean is now $$\operatorname{E}[Z \mid \mu = \mu_1] = \mu_1,$$ and therefore $$\begin{align*} \Pr[|Z| > z^* \mid \mu = \mu_1] &= 1 - \Pr[-z^* \le Z \le z^* \mid \mu = \mu_1] \\ &= 1 - \Pr\left[ -z^* - \frac{\mu_1}{\sigma/\sqrt{n}} \le \frac{\bar X - \mu_1}{\sigma/\sqrt{n}} \le z^* - \frac{\mu_1}{\sigma/\sqrt{n}} \mid \mu = \mu_1\right] \\ &= 1 - \left( \Phi\left(z^* - \frac{\mu_1}{\sigma/\sqrt{n}}\right) - \Phi\left(-z^* - \frac{\mu_1}{\sigma/\sqrt{n}}\right)\right), \end{align*}$$ where $\Phi$ is the cumulative distribution function of the standard normal.
A plot of this power function for a variety of hypothesized alternative means $\mu = \mu_1$ at the $\alpha = 0.05$ significance level (remember, this implies $z^* \approx 1.96$) is illustrated below. The horizontal axis is $\mu_1$, and the vertical axis is the power $1 - \beta$:
The power curve for $n = 10$ is shown in black. The animated colored curves represent an increase in sample size over the range $n = 1$ to $n = 50$. So as you can see, this makes sense: the power is greatest when the alternative mean is far from zero, and is least when the alternative mean is close to zero. Furthermore, as the sample size increases, the power to correctly reject $H_0$ also increases, and the range of alternative means for which we will be able to reject $H_0$ also improves, because the more data you collect, the better your chances at detecting if the underlying mean is in fact nonzero. But of course, as shown in all the curves, if the true mean IS zero, your test will never reject $H_0$ with a probability exceeding $\alpha = 0.05$, because that is your Type I error. This is represented by the $y$-intercept of the plot.
In light of the above discussion, I should take this opportunity to point out an important consequence of the structure of the hypothesis test. What I must impress upon the reader is that such a test can never validate $H_0$ and this fact is made especially evident by the power curve and its relationship to the Type I error $\alpha$. For we can see that irrespective of the sample size $n$, the curve always intersects the $y$-axis at $\alpha = 0.05$. That means that no matter how much data we collect, the test has a fixed probability of rejecting $H_0$ when $H_0$ is true, thus we can never use such a test statistic to furnish evidence $H_0$ is true.
This bears emphasis because it is shockingly common to see the misinterpretation of the $p$-value of such tests as indicative of the "truth" of $H_0$; that is to say, if the $p$-value is "high", this "suggests" $H_0$ is likely to be true. This is seriously flawed: the $p$-value, as determined by the test statistic, is a conditional probability, conditioned on the assumption that $H_0$ is true; thus it cannot tell you if $H_0$ is true, as this would be circular reasoning.
How, then, would you construct a statistic to demonstrate such a thing? Clearly, you would need a statistic whose power curve looks inverted compared to the one shown here: it would need to get larger the closer the alternative mean is to zero, and lower the further away it is. The description of such a statistic is beyond the scope of this response but for the interested reader, this is known as "equivalence testing" and typically involves a hypothesis of the form
$$H_0 : |\mu - \mu_0| \ge \Delta \quad \text{vs.} \quad H_1 : |\mu - \mu_0| < \Delta,$$ for some "equivalence margin" $\Delta$.