Consequences of mid-stream mode-switching in a panel survey

Abstract

Face-to-face (F2F) interviews produce population estimates that are widely regarded as the ‘gold standard’ in social research. Response rates tend to be higher with face-to-face interviews than other modes and face-to-face interviewers can exploit both spoken and visual information about the respondent’s performance to help assure high quality data. However, with marginal costs per respondent much higher for F2F than online data collection, survey researchers are looking for ways to exploit these lower costs with minimum loss of data quality. In panel studies, one way of doing this is to recruit probability samples F2F and subsequently switch data collection to web mode. In this paper, we examine the effect on data quality of inviting a subsample of respondents in a probability-based panel survey to complete interviews on the web instead of F2F. We use accuracy of respondents’ recall of facts and subjective states over a five-year period in the areas of health and employment as indicators of data quality with which we can compare switching and non-switching respondents. We find evidence of only small differences in recall accuracy across modes and attribute this mainly to selection effects rather than measurement effects.