Investigation Into the Effects of Using Normal Distribution Theory Methodology for Likert Scale Patient-Reported Outcome Data From Varying Underlying Distributions Including Floor/Ceiling Effects

Abstract

ObjectivesUtilization of parametric or nonparametric methods for testing Likert scale data is often debated. This 2-part simulation study aims to investigate the sampling distribution of various Likert scale distributions (including floor/ceiling effects) and analyze the effectiveness of using parametric versus nonparametric tests with varying sample sizes. MethodsWe simulated populations from parametric distributions binned into Likert scales. In study 1, replicates were sampled from each distribution with sizes ranging from 5 to 150 observations, calculating means with simulated 95% CIs at each sample size. In study 2, floor/ceiling effects were introduced such that the proportion of patients responding with the lowest rating varied from approximately 40% to 90%. Two-sample tests were then conducted for the 90% floor effect distribution against all other floor distributions to determine effectiveness of parametric versus nonparametric methods via 2-sided pooled t tests and Wilcoxon rank-sum tests. Coverage of the difference in means, realized P values, relative efficiency, measures of agreement in direction, and conclusion of tests were plotted by sample size. ResultsThe sampling distributions of the 1-sample means and SDs for most distributions converged quickly to Gaussian, with 95% coverage. One- and 2-sample t tests of the mean demonstrated acceptable coverage, type I error, and agreement. ConclusionsSimulations confirm that the sampling distribution of the mean rapidly approaches normality and appropriate tests provide adequate coverage and type I error. Two-sample t tests demonstrate appropriateness and increased statistical power gained by using parametric over nonparametric approaches, suggesting t tests should be implemented with few restrictions.