Statistic Power calculation

189 Views Asked by At

I recently started to study hypothesis testing. I find out that "The power of a hypothesis test is the probability that the decision rule leads to the right conclusion when the null is false."

so based on my search, I find that in order to calculate the probability of detection or power we should use the below formula

$$\int_ξ^\infty p(x|H1) \,dx$$

here x is our sample or data and H1 is alternative hypothesis and is threshold level

but when I tried to read another subject, such as proof of Neyman Pearson Lemma theorem, I saw that the power or power function defined as
$$E_\Theta[φ(X)]$$ ( φ is Test statistic and Theta is model's Parameters (Theta1 is parameter value when alternative hypothesis is true and Theta0 is model parameter when null hypothesis is the correct one)).

which I find confusing, so my question is: Are these two mentioned formulas the same?
and
How we calculate power based on which formula?

1

There are 1 best solutions below

2
On

You show formulas without definitions of symbols, and with no work of your own. So it is difficult to assess what you understand and what you don't. Here is a discussion of a very simple situation in which everything can be calculated using binomial distributions. I hope it is helpful.

Suppose you flip a coin $n=50$ times, suspecting it may be biased towards Tails. You will test $H_0: p = .5$ against $H_a: p \le .5,$ rejecting $H_0$ if the number $X$ of Heads in 50 independent tosses is less than or equal to critical value $c = 18,$

Because $P(X \le c|H_0) = P(X \le c|p=.5) = 0.0325$ this is a test at level 3..25%. That is, the probability of Type I Error is $ 0.0325.$ Because of the discreteness, it is not possible to do a (nonrandomized) test at exactly the 5% level of significance. [Computation in R, where pbinom is a binomial CDF.]

pbinom(18, 50, .5)
[1] 0.03245432

To find the power against the specific alternative $p_a = .25,$ you need to find $P(X \le c|p=.25) = 0.9713.$ That is, the probability of Type II error is 1 - 0.9713 = 0.0287.

pbinom(18, 50, .25)
[1] 0.9712668

A power curve for this test would plot the probability of rejection against the specific alternative values $p_a$ of interest. A plot of this power curve is shown below the R code used to make it. The dotted red line shows power at $p_a = .25,$ computed just above.

[To make a power curve with a calculator, you could plot half a dozen carefully chosen points $p_a$ (instead of 401), connecting them with a hand-drawn curve.]

p.a = seq(.5, .1, by=-.001)
pwr = pbinom(18, 50, p.a)
plot(p.a, pwr, type="l", ylim=0:1, lwd=2)
 abline(h = 0:1, col="green2")
 abline(v = c(.5,.1), col="green2")
 abline(v = .25, col="red", lty="dotted")

enter image description here