The Mode is the Message: Using Predata as Exclusion Restrictions to Evaluate Survey Design

Abstract

The authors consider how the mode of data collection (Internet vs. paper) alters individuals’ responses to different types of survey questions, including subjective, recall, and factual questions. The authors isolate the measurement effect of the mode from the sample selection effect by exploiting predata in a convenience consumer panel. The authors propose using panelists’ reward point balance as exclusion restriction to correct for differing response probabilities by mode, because the reward point balance depends on the timing of the survey invitations and is a source of random variation in response incentive. The authors evaluate average and quantile measurement effects in a mixed-mode Web/paper survey and find statistically significant evidence of mode effects in subjective and recall questions.