I am having trouble understanding hypothesis testing and p values, I have an idea of what they are but the trouble comes when applying those ideas to problems. I have some questions based on an example and I am just looking for explanations, not specific answers to the problem.
Let's say we are given two samples of data, gpa and salary each with $7$ pieces of data. For a $5\%$ significance level test the following hypothesis:
a. $H_o$ mean salary = $28$ against two sided alternative and both one sided alternatives.
First question: Here all I have to do is use the $t$ distribution (do not understand why) and find the point where the area is $5\%$ for each case, two sided tail and both one sided tails, then see where the observation (after standardizing, $\frac{\text{mean}-\mu}{\frac{\sigma}{\sqrt{7}}}$) is and decide to reject or not to reject the null hypothesis?
b. calculate the corresponding p values
Second question: I do not understand how to calculate the p value. Do I need to standardize the observation than use that number and see where it lies on the normal distribution and the p value is just the area under the curve, whether its for a two tailed, or either one tailed alternative?
c. calculate the power of the above tests for the following alternatives:mean salary = $25$, $30$, $35$
last question: now this is just asking to find the p value for $H_o=25$, $H_o=30$, and $H_o=35$, for two tailed and both one tailed alternatives?
I will try to help with some of this.
(a) Test $H_0: \mu = 28$ against two-sided alternative $H_a: \mu \ne 28.$ Presumably you have $n = 7$ observations and can find the sample mean $\bar X$ and sample SD $S.$ You will find the $T$ statistic. This would not be a t test if you knew $\sigma.$ Your test statistic is $$T = \frac{\bar X = 28}{S/\sqrt{7}},$$ which has Student's t distribution $T(7)$ with degrees of freedom $\nu = n-1 = 6.$ For a two-sided test at the 5% level, you will Reject $H_0$ if $|T| > t^*,$ where $t^*$ cuts probability 2.5% from the upper tail of $T(7)$. You can get this from a printed 't table' or using software: $t^* = 2.447.$ Sometimes the values $\pm t^* = \pm 2.447$ are called 'critical values'. If $T$ falls outside $(-2.447, 2.447)$, then you reject the null hypothesis.
For example, suppose $\bar X = 32.2$, $S = 4.40$, and $T = 3.27.$ Then you reject the null hypothesis at the 5% level of significance because $|T| = 3.27 > t^* = 2.447.$ Intuitively, $\bar X$ is a 'good' estimate of $\mu$ and we we will reject $\mu = 28$ as a believable value if $\bar X$ is 'too far from' 28. Quantitatively, 'far from' is measured by $T$, and $|T| > 2.447$ means it's 'too far'.
(b) The P-value is the probability of a result as extreme or more extreme than the value observed. For a two-sided test, 'extreme' means 'different in either direction'. If the null hypothesis is true, we expect $T$ to be 0. You have observed $T = 3.27.$ So the P-value is $P(T > 3.27) + P(T < -3.27).$ The second term is because -3.27 is just as far from 0 as is 3.27, but in the opposite direction. Finding the P-value for a t test is a matter for software. You haven't said anything about the level of your course. So I don't know how much detail to give.
Minitab software gives the following printout, showing the P-value.
You can tell whether to reject at the 5% level just by looking at the P-value. Reject if it's less than 5%. It is (0.017 < 0.05$, so reject.
In R statistical software you could compute the P-value as follows:
Here is a graph of $T(6)$. The vertical red lines at $\pm 2.447$ enclose 95% of the area under the density curve, leaving 2.5% in each tail and a total of 5% in both tails. The solid green line shows the observed value of $T = 3.27$ and the broken green green line at -3.27 is just as extreme as 3.27 in the other direction. The P-value is the total area in the two tails out beyond the green lines (about 0.0085 in each tail).
(c) Power. The power of the test for a particular alternative is the probability of rejecting if $\mu_a$ takes some value other than the hypothetical value $\mu_0 = 28.$ We speak of a power function because there are many ways for $H_0$ to be false and there is a different power for each of them. Without knowing the level of your course, I really can't tell you how to find the power function. Technically, it depends on the 'non-central t distribution'.
Minitab has a procedure that makes power curves, when you provide all the necessary information. One piece of necessary information is the population SD $\sigma.$ I'll just use the estimate $S = 3.40.$ Other information is the distance(s) from the null value. You were asked for $\mu_a = 25, 30, 35$ so I used differences -3, 2, and 7.
Notice that the power is about 99.4% for $\mu_a = 35$ (difference of 7 from $\mu_0 = 28.$ So if the true population mean were $\mu = 35$ a two-sided t test with $n = 7$ observations would have a very high probability of rejecting $H_0: \mu = 28.$
One-sided tests. If you are testing $H_0: \mu = 28$ against $H_a: \mu > 28,$ there will be only one critical value $t* = 1.943.$ Also, the P-value will only consider the area to the right side of the observed value $T$.
I will let you figure that out for yourself. If you edit your question to show some additional engagement with this, and have additional questions to ask, perhaps you will get additional answers. It would be really helpful if you said something about the level of the course and the text.