A hypothetical HIV vaccine trial involving 20,000 participants—10,000 in the vaccine group and 10,000 in the placebo group—had the following results: 6.3 infections per 1000 in the vaccine group and 9.0 infections per 1000 in the placebo group.
I ran a computer simulation to predict possible outcomes of the trial if the null hypothesis is true—that is, if vaccinated and unvaccinated people are equally likely to contract HIV. I ran 1000 virtual trials of 20,000 people (10,000 per group) assuming that the vaccine is ineffective. Outcomes are expressed as excess infections in the placebo group. Here are the results of the 1000 virtual trials displayed as a histogram.
- Roughly estimate the (two-sided) p-value associated with the trial’s outcome from the histogram.
2.From the simulation, I learn that the statistic ‘excess infections in the placebo group’ follows a normal distribution with a mean of 0 and a standard deviation (standard error) of 12.3. Use this information to calculate the two-sided p-value more precisely than in (1).
For 1) if you've simulated the excess infections under the null then you have numerically estimated the sampling distribution. Just calculate the fraction of the simulation results that are $\geq 3.3$ or $\leq -3.3$ This is the two-sided p-value for your observed placebo excess.
For 2) you are not using your simulation, but the $\mathcal{N}(0,12.3)$ distribution in lieu of the simulation results. Now, just calculate $2(1-\Phi(\frac{3.3}{12.3}))$. This is based on the normal distribution.
Compare results.