Likert scale analysis

Question

I have collected survey using a 5 point Likert scale (ranging from very negative affect to very positive affect). I am writing a scientific paper so wish to follow similar procedures.

Is it appropriate to provide statistical analysis to compare the 'mean' (I know that the idea of a mean in Likert data is contentious in itself) response to 'no impact'? For instance, if 75% of respondents have answered 'somewhat positive affect' and 25% have answered 'very positive affect', is it a requirement/desireable to use a statistical test (such as the one sample sign test) to compare this test this against a null hypothesis of the 'mean' response being 'no impact'?

Thanks in advance :)

Answer 1

For a talk and perhaps for a figure in a forthcoming paper, I would suggest you show a pie chart: Color 'Strongly Pos' deep cyan, 'Pos' light cyan, 'No' grey, "Neg' peach, 'Strongly neg' deep orange. Then note that cyan areas (some kind of positive) take up far more than half of the pie (71% of respondents). Also, because there are $n = 50$ respondents a 95% Agresti-style ("plue-four") confidence interval for "overall positive" is $(.56,.81),$ so we're pretty sure the exact population proportion of "overall positive" exceeds half. [There are some approximations computing the CI, because 53% of 50 isn't an integer.]

I do not see how you get P-value 0.035. First, intuitively that seems a 'weak' P-value (rejecting at the 5% level, but not at the 3% level), while the data seem extremely strongly positive. More than half of the results are 'Strong positive' or 'Positive' responses, so the observed median response is 'Positive'.

If I have understood your data correctly, here are results of Minitab's output for a sign test (coding categories as 2=Strongly pos, 1=Positive, 0=None, -1=Negative).

Sign Test for Median: C1 

Sign test of median =  0.00000 versus ≠ 0.00000

     N  Below  Equal  Above       P  Median
x   50      6      9     35  0.0000   1.000

The tiny P-value indicates that if the true opinion were 'None', then it would be almost impossible to get as many 'Strong pos' and 'Positive' responses out of 50 as you observed.

I think a permutation test would be more powerful than a sign test for data of the general kind you have, especially with smaller $n$'s But for the data you give in your Comment, there is no point doing a permutation test because the sign test gives such a tiny P-value.

Please take another look at your data counts, and let me know what you find. Maybe you have given wrong counts, or maybe I have misunderstood what you said.