Mode Effects on Variances, Covariances, Standard Deviations, and Correlations

Abstract

The European Social Survey (ESS) program currently includes an investigation of mixed-mode survey designs, in which different respondents complete the survey by different survey modes. Such mixed-mode designs may lower systematic and random selection error if selection effects occur between the modes. Nonetheless, the advantage of selection effects may be counteracted by measurement effects between the modes. In the existing literature, selection and measurement effects in mixed-mode designs have already been extensively researched for mean and proportion parameters. Surprisingly, both effects have hardly been researched for covariance parameters, even though covariances are sufficient statistics for widely used parameters such as correlations, regression coefficients, or and factor loadings. This paper analyzes measurement and selection effects on variances, covariances, standard deviations, and correlations for different items from MTMM experiments within the Estonian European Social Survey Round 6 data by using the instrumental variable method. This analysis yields small to moderate selection and measurement effects, which means that mixed-mode surveys may perform slightly better than single-mode surveys, but also that analysis of mixed-mode data may require small error correction for measurement effects.