Evaluating Data

Question

I have a set of data that I need to evaluate. The data came from 15 people we asked to evaluate several project proposals using 6 different criteria. For each weighted criteria, they selected 1 of 4 choices, each choice given a value of 0, 3, 6 or 9.

Criteria Example: What is the value to the customer:

0= Little Value 3= Some Value 6= A lot of value 9= Significant/Critical value to customer

While I expected to see some differences in scores, I am concerned at the varied results. I am trying to find a way numerically represent my concern.

I have researched across the web looking for an understanding and interpretation of Standard Deviation, I am not sure if this is the right function to use.

Looking forward to discussion, learning and answers :)

---

Before receiving the flaming responses, we can agree to disagree that I am stupid or an idiot, or that I should have read through other peoples questions. I have reviewed other posts and while there is good information, I am not finding the information I seek. I am a quick learner, emphasis on the word learner.

Darcy

Answer 1

First, don't be down on yourself!! I agree with you that standard deviation is an excellent measure of what you want to quantify. It is a measure of the variation, or how "spread out" the data is. Alternatively (but equivalently), you can use the variance.

Answer 2

Since you have 15 people, it is not too bad to use a Gaussian approximation to the average score assigned by people for a particular category. Calculate the average of the scores for a category over the 15 customers, and then compute the standard deviation of the 15 scores for the category. Then your statistic for determining whether the score is unusually high or low is the average minus (0+3+6+9)/4 = 18/4, then divided by the standard deviation, then multiplied by square-root of 15. You can use this as a so-called "z-score" for a Gaussian variable and convert the z-score into a p-value (probability that this result or more extreme occurred by random chance) using a z-score look-up table or a computer function that converts z-score into p-value. When you get a low p-value (say 0.05 or less), you can be confident that the average score assigned by people was statistically significantly high or low. When the p-value is higher, then the result for the category could have occurred by chance. If you want a potentially even more informative test, then take the average score for a category minus the average score over all categories, instead of the average score for a category minus 18/4.