Subjective Response Differences between Visual Analogue, Ordinal and Hybrid Response Scales

Abstract

The Human Factors and Ergonomics (HF/E) discipline employs different subjective response scale formats to measure subjective phenomena (e.g., hazard perception). Per the psychometrics literature, different scale formats can yield different participant responses, which is a potential threat to validity in replication studies if response scales are not consistent across study iterations. If ordinal response scales (e.g., Likert scales) yield ordinal data and continuous response scales (e.g., Visual Analogue Scales) yield continuous data then it is inappropriate to substitute one response scale for the other if they potentially yield different responses for the same question. The current research compared mean participant ratings for the same question on VAS, Likert and Hybrid response scales and found that VAS scales had significantly lower mean ratings than Likert and Hybrid response scales. Two scale features, the number of anchors (5 or 9) and the scale length (10.0cm and 19.2cm), were varied and no significant main effects or interactions resulted. In conclusion, scale types, not scale features, produced significant mean participant rating differences. To support the validity of replication research, this paper also provided a response scale taxonomy based on the scale features studied herein, that can be used to classify and report different response scales. Implications for these results and future research directions are discussed.