Differential Effects of Web-Based and Paper-Based Administration of Questionnaire Research Instruments in Authentic Contexts-of-Use

Abstract

Questionnaire instruments are routinely translated to digital administration systems; however, few studies have compared the differential effects of these administrative methods, and fewer yet in authentic contexts-of-use. In this study, 326 university students were randomly assigned to one of two administration conditions, paper-based (PBA) or web-based (WBA), and given a set of questionnaires. Instructions were to complete the instruments in an environment of their choice, and data included reporting context characteristics. Outcomes of interest included data quality and participant affect—WBA showing a slightly higher percent of data loss and lower overall time to complete; PBA producing higher overall mean scores across measures, greater variability in responses, and higher positive affect for responding. Administration methods showed no difference on internal consistency of subscales, positive-response bias, or strength of interscale correlations. Contexts-of-use included involvement in television viewing, conversation, and other activities, raising questions about the accuracy and independence of survey responses completed in independently-chosen, uncontrolled contexts. The qualitative data demonstrated longer responses in WBA than PBA, but little difference in type and clarity of responses.