Do Sequential Mixed-Mode Surveys Decrease Nonresponse Bias, Measurement Error Bias, and Total Bias? An Experimental Study

Abstract

Abstract Mixing multiple modes of survey data collection has become standard practice in survey research. Mixed-mode surveys are faced with a slew of design decisions regarding which types of modes to administer and which sequence to administer them in. Such decisions are largely based on administrative objectives, such as minimizing costs and maximizing response rates. However, just as important to these mixed-mode decisions is their impact on nonresponse bias, measurement error bias, and total bias, which are understudied issues in the mixed-mode literature. In this article, we report on a sequential mixed-mode experiment of young adult drivers randomized to one of two mode sequences: an interviewer-administered (telephone) mode with self-administered (mail) follow-up, or the reverse sequence. Using a mix of direct and indirect bias estimation strategies, we find support for the notion that implementing a second mode of data collection can reduce nonresponse and measurement error bias, but the sequence in which the modes are administered makes a difference: the mail-telephone sequence minimizes bias to a greater extent than the telephone-mail sequence, relative to the starting mode and overall. However, a backfiring effect was found: despite reducing both nonresponse and measurement error bias, switching from mail to telephone increased the total bias in a key estimate of traffic accidents. A discussion of these findings and their implications for survey practice are provided in conclusion.