Moving from Face-to-Face to a Web Panel: Impacts on Measurement Quality

Abstract

Abstract Time and cost pressures, the availability of alternative sources of data, and societal changes are leading to a move from traditional face-to-face surveys to web or mixed-mode data collection. While we know that there are mode differences between web and face-to-face (presence of an interviewer or not, type of stimuli, etc.), it is not clear to what extent these differences could threaten the comparability of data collected in face-to-face and web surveys. In this article, we investigate the differences in measurement quality between the European Social Survey (ESS) Round 8 and the CROss-National Online Survey (CRONOS) panel. We address three main research questions: (1) Do we observe differences in terms of measurement quality across face-to-face and web for the same people and questions? (2) Can we explain individual-level differences in data quality using respondents’ characteristics? and (3) Does measurement equivalence (metric and scalar) hold across the ESS Round 8 and the CRONOS panel? The results suggest that: (1) in terms of data quality, the measurement mode effect between web and face-to-face as implemented in the ESS (i.e., using show cards) is not very large, (2) none of the variables considered consistently explain individual differences in mode effects, and (3) measurement equivalence often holds for the topics studied.