False Respondents in Web Surveys

Abstract

Purpose: The aim of the conducted analyses was to propose and test an FR procedure for detecting false respondents (who answer survey questions mindlessly) in online surveys. Design/methodology/approach: Statistical analyses of data from 9 online surveys with a total of 4224 respondents, and 3 offline surveys (a total of 3169 respondents), aimed to identify false respondents using 4 warning signs (WS) based on: (WS1) too short answering time, (WS2) attention check questions, (WS3) rating style that considers, among others, the number of “Don’t know”, (WS4) logical consistency test of the answers and self-reported engagement of respondents. Findings: The percentage of respondents flagged by any of 4 signs (strict criteria) ranged from 5.2% to 71% depending on the survey. With lenient criteria (allowing respondents to be flagged by one warning sign), the percentage of excluded respondents ranged from 0% to 45.9%. Respondents could be excluded from analyses locally (for a specific block of items) or globally. Research limitations/implications: The surveys used in the analyses in this paper were of high quality (designed to minimize the participation of false respondents), which means that the percentages of false respondents for surveys made available to all interested parties will be higher. The analyzed data included respondents with at least secondary education. Originality/value: The conducted analyses provide evidence for the necessity of cleaning data obtained in online surveys. The tested FR procedure proved to be useful. The utility of the FLEXMIX procedure for examining the logical consistency of respondents’ answers was also demonstrated.