Make some fake unpaired Likert data, plot it, and analyze it with some tests. This pretends that you have two groups (A and B), with 60 people in each group, and your survey has 7 questions, each scored on a 0 to 4 Likert scale.
Below is a table of p values, comparing group A to group B, for each of the questions. The following statistical tests (columns in the table) are tried:
- Wilcoxon rank sum, a.k.a. Mann-Whitney
- T-test (unpaired)
- Chi-squared test for trend in proportions (Cochran–Armitage test for trend)
- Pearson's chi-squared test ("plain old" chi-squared that doesn't know ordinal from categorical)
Some authors say that the t-test might be more appropriate than it seems. See Norman and Sullivan.
question wilcoxon_p ttest_p catt_p chi_p
1 0.00008602 0.00001416 0.00002332 0.00001884
2 0.00000000 0.00000000 0.00000000 0.00000003
3 0.00000000 0.00000000 0.00000001 0.00000028
4 0.00090077 0.00037737 0.00045219 0.00241743
5 0.00327405 0.00119031 0.00131765 0.01453107
6 0.61915745 0.66912503 0.66595722 0.95185781
7 0.07541019 0.09881224 0.09722275 0.10866653
Plot of fake survey data (one question per row):
