Trying to understand what is a p value

Question

I am trying to understand what a p value is and what is its importance in statistics, from this example.

For $50$ specimens of alloy steel the rock hardness scale measured an average of $62$ with standard deviation $8$ he manufacturer claims that the alloy has average strength of $64$ at significance of $0.01$ can you refute this?

Well I did this problem and saw no at this level you cant refute it , I use the null being that the hardness was $64$ and the alternate that the hardness was less then $64$. From the data we observe a $z=-1.7677$ but the chart is $z=-2.33$ and so we cant refute null at this level.

But now I want to understand what the p value would be ,

I thought it was that the p-value is the probability of getting the observed value of the test or a value with even more evidence against the null if the null is actually true. So using that I just look on the z chart, the probability of seeing Z greater then $1.7677$ is $0.0384$, so is that the p value?

Answer 1

First off, if you have the sample standard deviation, not the population standard deviation, you should run a $t$-test, not a $z$-test. However, I'll assume you’re assuming that the population standard deviation is also $8$, which in practice is an unwarranted assumption, but it’s okay while you’re learning the material so you don’t have to mess with $t$-tables.

The wording of the question is a bit odd; the manufacturer’s claim is not based on the sample, so a better statement of the question would be this:

A manufacturer claims that a steel alloy it produces has an average hardness of 64. In order to evaluate this claim, you test a random sample of 50 specimens, which have an average hardness of 62 with standard deviation 8. Assuming that the population standard deviation is also 8, does this refute the manufacturer’s claim at a significance level of 0.01?

Unless you have reason to believe that the alloy hardness is specifically less than 64, you should run a two-tailed test. The $p$-value for this test turns out to be 0.0771. This is the area under the distribution of sample means that contains sample means as far or further from 64 than your sample (in either direction).

(Note: for a $t$-test, it turns out to be 0.0833.) For a one-tailed test, it would be 0.0771/2, or just the area corresponding to sample means further from 64 in the direction of your sample mean.

This is what you found (close enough, up to rounding), so the short answer to your question is that you have correctly understood the $p$-value.

Answer 2

This Wikipedia article is actually very useful, in that it describes all of the points and misconceptions of what a p-value actually is:

https://en.wikipedia.org/wiki/P-value

If you look at the table from which you are reading the "Critical Values," denoted by z, there will almost always be a graph at the top. This graph is the Normal Distribution. Here is a picture of it. The critical value, -2.33, is the value of x for which the following is true:

$$\int_{-\infty}^x Normal.Distribution[\mu=0,\sigma=1] = 0.01$$

That is, if we integrate the Probability Distribution Function (PDF) of the Normal Distribution from negative infinity to x, then we get 0.01 as the output.

Typically, we don't know what distribution the data particularly belong to, but in many introductory statistics courses, the Normal Distribution is used, with the Central Limit Theorem as justification. The Student's T-Distribution is an alternative method when the population standard deviation is unknown.

That aside, even if we assume the data belongs to one of these distributions, an observation doesn't necessarily mean anything when looking at the Standard Normal Distribution (a Normal Distribution with a mean of 0 and standard deviation of 1). Thus, we have to calculate the Z-statistic, which scales the result we're testing (hardness=64, your null hypothesis) into something that resembles an x-value for the Standard Normal Distribution. You calculated it when answering the question:

$$z_* = \frac{x-\mu}{\sigma}$$

We take this "standard-score" and apply it in the same way we applied the critical value:

$$\int_{-\infty}^{z_*} Normal.Distribution[\mu=0,\sigma=1] = p$$

The p in this equation is the p-value; rather, the probability of seeing this occur given the null-hypothesis is true. If it is greater than the amount we are willing to be wrong, denoted by $\alpha$ (0.01 in this case), than we fail to reject the null-hypothesis (p>$\alpha$). However, if p<$\alpha$, then we reject the null-hypothesis. This is also why you can simply compare the standard-score to the z-critical value, rather than compute the p-value.

Does this help your understanding?