Does Giving People Their Preferred Survey Mode Actually Increase Survey Participation Rates? An Experimental Examination

Abstract

Survey research has long grappled with the concept of survey mode preference: the idea that a respondent may prefer to participate in one survey mode over another. This article experimentally examines the effect of mode preference on response, contact, and cooperation rates; mode choice; and data collection efficiency. Respondents to a 2008 telephone survey (n = 1,811; AAPOR RR3 = 38 percent) were asked their mode preference for future survey participation. These respondents were subsequently followed up in 2009 with two independent survey requests. The first follow-up survey request was another telephone survey (n = 548; AAPOR RR2 = 55.5 percent). In the second follow-up survey (n = 565; AAPOR RR2 = 46.0 percent), respondents were randomly assigned to one of four mode treatments: Web only, mail only, Web followed by mail, and mail followed by Web. We find that mode preference predicts participation in Web and phone modes, cooperation in phone mode (where contact and cooperation can be disentangled), and the selection of a mode when given the option of two modes. We find weak and mixed evidence about the relationship between mode preference and reduction of field effort. We discuss the important implications these findings have for mixed mode surveys.