As a school, we ask our pupils to complete Progress Towards Maths (PTM) and Progress Towards English (PTE) tests. From this, we get generate Stanines for each pupil. I am looking to replicate this representation using a pupil's summer exam results.
Clarity of 'replicating representation': Use the end-of-year exam results and translate them (using the mathematical process) into a stanine to provide an idea of where that pupil is placed within the year group.
What the data will be used for: I am hoping to use this data to track a pupil's progress from when they enter our school at the age of 11 and potentially predict (with some accuracy) their GCSEs
The process I have is as follows:
- Get a list of raw scores for each subject
- Find the mean of all scores
- Find the difference between the pupil score and the mean
- Square the difference, find the sum of that across the list of pupils and divide it by the number of pupils there are to get the quotient, then find the square root of that value.
- Produce a z-score by dividing the value in step 3 by the value in step 4.
- Depending on that value each pupil is given a stanine based on the following: Stanine 1 consists of z-scores below -1.75; stanine 2 is -1.75 to -1.25; stanine 3 is -1.25 to -0.75; stanine 4 is -0.75 to -0.25; stanine 5 is -0.25 to 0.25; stanine 6 is 0.25 to 0.75; stanine 7 is 0.75 to 1.25; stanine 8 is 1.25 to 1.5; and stanine 9 is above 1.75.
That is the process I am using. Due to a range of circumstances, some pupils may miss some exams. Do I include those pupils in the calculations as a 0 mark? Does this have any advantages or disadvantages?
My thoughts are to leave them in as it shows a pupil's distribution across the whole year group, however, if the pupil did not actually sit the exam then the result does not reflect the pupil and therefore impacts (inflates?) the results of others?
EDITED FOR MORE THOUGHTS
More detail from my thoughts: Firstly this was a last-minute idea and collection of KS3 data and moving forward I would include multiple tests rather than an end-of-year exam (much like GCSE). If a pupil in the top class does not sit the exam then they are marked as 0/100 score. If a pupil in the bottom class does sit the exam and scores 50/100 then that lower pupil would be placed above all pupils scoring 0 thus the representation of that pupil in the year group would not be a true reflection had the top class pupil actually sat the exam. It would be a reflection against that one test but not for an end-of-year summative result. Does that result actually tell us anything then?
For the sake of this response, I'm going to assume that your procedure for mapping raw scores into stanines is basically correct. I've spot checked it, but haven't rigorously checked it. The problem about what to do with the student's who did not sit for the test is what I'm going to address.
The issue with the blank scores is called a missing data problem. There's no hard-and-fast correct way to deal with it. But there are various strategies depending on what you want your scores to represent.
If you're just trying to get overall numbers or stanines to represent students who actually took the test, then I'd say just dispense with the 0s of any student who did not sit for the exam. If you consider the fact that some students did not sit for the exam to be just as representative of the overall group as any score of any student who actually sat for the exam, then I'd say go ahead and include the 0 scores. Simply put, you have to use your judgement about what you're trying to show about these students to decide what to do.
You are also correct for recognizing that including these scores without addressing the fact that they didn't sit for the exam in the first place may skew your conclusions about all kinds of things.
If you're wishing to prove some point about the students who didn't sit for the exam, or differences between those who did and those who didn't, there are various strategies for tackling these questions called data imputation. But I'm thinking imputation is a topic for a different discussion.
Feel free to email me at the address above if you want to discuss the subject further.