Investigating respondent multitasking in web surveys using paradata

Abstract

Computers play an important role in everyday multitasking. Within this context, we focus on respondent multitasking (RM) in web surveys. RM occurs when users engage in other activities while responding to a web survey questionnaire. The conceptual framework is built on existing literature on multitasking, integrating knowledge from both cognitive psychology and survey methodology. Our main contribution is a new approach for measuring RM in web surveys, which involves an innovative use of the different types of paradata defined as non-reactive electronic tracks concerning respondents' process of answering the web questionnaire. In addition to using questionnaire page completion time as a measure of RM, we introduce ‘focus-out’ events that indicate when respondents have left the window containing the web questionnaire (e.g., to chat, email, browse) and then returned. The approach was tested in an empirical study using a web survey on a student sample (n = 267). The results indicate that 60% of respondents have multitasked at least once. In addition, they reveal that item nonresponse as an indicator of response quality is associated with RM, while non-differentiation is not. Although this study confirms that a paradata-based approach is a feasible means of measuring RM, future research on this topic is warranted.