Reassessing the Effect of Survey Characteristics on Common Method Bias in Emotional and Social Intelligence Competencies Assessment

Abstract

Since the idea of method variance was inspired by D. T. Campbell and Fiske in 1959, many papers have demonstrated an ongoing debate about both its nature and impact. Often, method variance entails an upward bias in correlations among observed variables—common method bias. This article reports a split-ballot multitrait–multimethod experimental design for estimating 2 opposite biases: the upward biasing method variance from the reaction to the length of the response scale and the position of the survey items in the questionnaire and the downward biasing effect of poor data quality. The data are derived from self-reported behavior related to emotional and social competencies. This article illustrates a methodology to estimate common method bias and its components: common method scale variance, common method occasion variance, and the attenuation effect due to measurement errors. The results show that common method variance has a much smaller impact than random and systematic measurement errors. The results also corroborate previous findings: the greater reliability of longer scales and the lower reliability of items placed toward the end of the survey.