A Mixed-Mode and Incentive Experiment Using Administrative Data

Abstract

Abstract This research note compares sequential and concurrent web-mail mixed-mode approaches and incentives for a survey on US consumers’ use of financial products, especially when their finances are tight. The sample (n = 2,000) was drawn from credit bureau data. We examine the effects on response rates, survey costs, and possible nonresponse bias in an experiment varying two factors in a 2x2 design: (1) using concurrent or sequential web and then mail survey modes, and (2) different incentive amounts given to initial survey nonrespondents ($5 versus $10). The sequential (web-first) design had a significantly lower response rate (3.9 percent) at week five—before the paper questionnaire was mailed—than the concurrent group (11.0 percent). This difference was nearly fully eliminated by the end of the field period. The higher incentive brought in slightly more respondents in the concurrent arm and slightly fewer respondents in the sequential arm, but neither difference is statistically significant. Compared with the sample frame, respondents in both groups were generally older and had many of the characteristics that come from being older: higher credit scores, more open credit cards, lower credit card utilization, greater likelihood of having a mortgage, and lower likelihood of being delinquent on credit card payments. Given the lower initial response rate and the need for more follow-up mailing, the sequential mixed-mode approach resulted in a higher cost per complete survey.