How to test a hypothesis which compares set of pairs of statements?

52 Views Asked by At

I've conducted an experiment but I'm not sure how to proceed with statistical analysis of it.

I have pairs of sentences created by two groups of people A and B, semantically the sentences in each pair are the same but the grammar is little bit different. The participants of the experiment had to decide which sentence from each pair sounds better. They could say that A is much better than B, A is slightly better than B, they are indistinguishable, B is slightly better than A, B is much better than A- 5 options to choose from.

I have 20 pairs of sentences graded by 20 participants.

I would like to check if it is possible to say that the sentences from group A sounds not worse than the sentences from group B (so the sentences from A were graded as much better, slightly better or indistinguishable from B).

I guess I should start with the normality test, like QQ plots or Shapiro-Wilk, right? But I'm not sure how to proceed with the data I have.

Any which tests later I should use?