Straightlining: Overview of Measurement, Comparison of Indicators, and Effects in Mail–Web Mixed-Mode Surveys

Abstract

Straightlining occurs when survey respondents give identical (or nearly identical) answers to items in a battery of questions using the same response scale, which may reduce data quality. Despite its potential importance, research examining straightlining does not use a standard measurement technique. Further, while mixed-mode studies are increasing in prevalence, few studies compare straightlining behavior in mail versus web surveys. Our article has the following goals: (1) describe and evaluate methods for detecting straightlining and (2) examine effects of education and mode of administration on straightlining in two mail/web mixed-mode surveys. Data for Study 1 are from a 2010 survey of alcohol beliefs and consumption in which 7,200 young adults were sampled from driver’s license records in Wisconsin and randomly assigned to mail–web (mail followed by web) or web–mail (web followed by mail) treatments. Data for Study 2 are from a 2013 survey about a public university and its affiliated health organizations that used an address-based sample and randomly assigned households to one of the three experimental groups: mail-only, web-only, and web–mail. We identify and examine five methods for measuring straightlining: simple nondifferentiation, mean root of pairs, maximum identical rating, standard deviation of battery, and scale point variation. The overall results replicate previously reported findings of a negative association between education and straightlining behaviors except for the standard deviation of battery measure in Study 1. Controlling for gender, race/ethnicity, and education, mode of administration was not significantly related to straightlining for any of the measures, suggesting straightlining behavior is stable across mail and web forms of self-administration.