Professional Respondents in Opt-in Online Panels: What Do We Really Know?

Abstract

Survey researchers often assume that “professional” respondents, those who complete a large number of surveys in opt-in online panels, are more likely than others to provide low-quality responses because their primary motivation is to earn rewards with minimal effort. However, there is little empirical evidence for this assumption. It could also be that professional respondents are willing to expend effort in order to be compensated. We investigated this issue using data from four independent surveys of opt-in panelists with about 2,400 respondents in each survey. We classified panelists into three groups—"professional", “average,” and “novice”—according to the number of surveys they had previously completed and the number of panels they belonged to. We then compared the groups with respect to their demographic characteristics, reasons for joining a panel, and response behaviors. Professional respondents were the oldest group and, as expected, the most likely to report “for money” as the main reason for joining a panel. However, novices were actually the most likely to provide low-quality responses. Professional respondents appear to take the task of completing surveys more seriously than previously thought.