Not to Be Considered Harmful

Abstract

The number of respondents who access web surveys on a mobile device (smartphone or tablet) has been increasing rapidly over the last few years. Compared with desktop computers, mobile devices have smaller screens, different input options, and are used in a larger variety of locations and situations. The suspicion that the quality of data may suffer when online respondents use mobile devices has stimulated a growing body of research, which has mainly focused on paradata and web survey design. To investigate whether the respondents’ device affects the quality of web survey data, we examined the responses of 1,826 mobile-device and desktop participants in a political online survey that asked questions about the 2013 German federal election. To determine the reliability and validity of data submitted via mobile devices, we determined the consistency of the participants’ responses across questions and validated the responses against various internal and external criteria. Replicating previous findings, mobile-device respondents were younger and more likely to be female, and they produced higher dropout rates and longer completion times than desktop respondents. However, data produced by respondents using mobile devices were as consistent, reliable, and valid as data produced by respondents using desktop computers. These findings contradict the notion that mobile-device users compromise the reliability and validity of data collected online and suggest that researchers do not necessarily need to be afraid of the participation of mobile-device respondents in web surveys.