Response Order Effects in Online Surveys

Abstract

Online surveys are fast becoming the favourite tools of researchers due to various advantages they offer like ease of administration, reduced survey cost and real time data analysis. However they also have their drawbacks like any survey method; some of them being lack of population representativeness due to digital divide, consideration of the survey as spam eliciting low responses and technical glitches. Survey errors are generally classified in to two groups of Sampling & non-Sampling errors. Respondent errors, one of the most widely studied non- Sampling errors consist of Primacy and recency effects. The response order effects both Primacy and Recency have not been studied in online surveys according to accessible literatures; though their effect in oral and paper -pencil surveys are well documented. The existence of respondent errors in online surveys may vary vastly from other survey methods due to the respondent characteristics like higher education levels, control on the response setting and time and independence in response in the absence of a survey administrator. Using an experimental design we examined and provided preliminary evidence for response order effect (Primacy effect) in online surveys. We have also explored the impact of question complexity and questionnaire length on response order effects. The results indicate the existence of response order effects in online surveys, much like the traditional methods. The result also indicates the exacerbation of response errors due to question complexity and questionnaire length. Our study may offer valid insights and ideas to survey researchers, who use online survey tools, to reduce response order effects and thereby make their survey results more accurate.