Choir of believers? Experimental and longitudinal evidence on survey participation, response bias, and public service motivation

Abstract

Collecting data through surveys are a vital method in public administration research, but an overseen challenge for the validity of survey research is response bias, which is when response behavior correlates with study outcomes. Response bias can be particularly threatening in motivation research because motivation is an important determinant for behavior and most likely also for survey participation. Specifically, public service motivation (PSM) can increase the willingness to devote time and effort to respond to surveys for the benefit of others, whereas extrinsically motivated individuals may be less inclined to participate in surveys without prospects of economic compensation. This article examines motivation and response bias by use of a preregistered field experiment and two panel datasets. The experimental study shows that a monetary incentive increases the response rate, whereas a PSM-oriented appeal does not. Furthermore, the increased response rate of the incentive is achieved without any detectable response bias on PSM. Likewise, the panel studies offer little and mixed support for response bias related with PSM. This evidence suggests that (1) PSM is not a (strong) driver of response bias, and that (2) extrinsic motivational tools applied to PSM surveys are not necessarily associated with response bias.