I have around 100 students who have all sat 3 exams and are about to sit another. Since their last exam I have made changes to my instruction and I want to determine if any changes in student scores is statistically significant.
I took one statistics class in university, but I can't remember exactly how to calculate the p values, assuming that's the best way to determine statistical significance.
Thank you in advance.
Any test that simply looks at just the students who underwent the change in instruction isn't really valid. It would need to be compared with a before and after test of a similar number of students who didn't receive a change in instruction (a control group). Otherwise, how do you know the test wasn't simply easier? There are also other criteria for a valid test like randomly assigning treatment and control groups, double blind testing where the students and the graders don't know which group is which.
You could then run a test comparing a difference in means between the two groups. Take the mean and standard deviation of each groups' difference in scores and run a $2$ sample t-test.
$$t = \frac{\bar x_1 - \bar x_2}{\sqrt{\frac{s^2_1}{n_1}+\frac{s^2_2}{n_2}}}$$
Where $\bar x$ is the mean of the difference in scores from before and after modified instruction tests, $s$ is the standard deviation of the differences and $n$ is the sample size.
Then look at a t-table to determine a p-value and compare it with an appropriate level of significance $(.05)$. A p-value less than $.05$ would be statistically significant.