Mode effects on socially desirable responding in web surveys compared to face-to-face and telephone surveys

Abstract

This paper elaborates upon differences in socially desirable responding as being the result of mode effects between web, telephone, and face-to-face survey modes. Social desirability is one of the main threats to comparability of data between different modes. The paper conceptualises socially desirable responding as a specific type of mode effect, which is not only a result of inherent characteristics of a survey mode, but is also mediated and moderated by complex interdependencies of specific survey implementations, contextual factors, and characteristics and behaviours of respondents. While web surveys are generally less prone to socially desirable responding, it is essential to be wary of circumstances that may reduce the perceived privacy of the survey situation and lead to biased reporting. The presented empirical study analyses the answers to a large number of items used in a pilot implementation of the Generations and Gender Survey across the three modes to gain insights into the incidence of socially desirable responding and its role in the observed differences in estimates. The comparison of means, distributions, and proportions of extreme responses to scale questions is performed across 89 survey items. The results are in line with the previous findings on lower susceptibility of web surveys to social desirability bias. More importantly, the findings suggest that the problem of socially desirable responding is likely to be a major contributor to the differences in mean estimates, response distributions, and the level of extreme responding between the studied modes.