Reliability and Structure of the TALIS Social Desirability Scale: An Assessment Based on Item Response Theory

Abstract

One of the ways of controlling for the influence of social expectations on the answers given by survey respondents is to use a social desirability scale together with the main questions. The social desirability scale, which was included in the Teaching and Learning International Survey (TALIS) international comparative study for this purpose, was used on a Russian-language sample of teachers without cross-cultural adaptation. In addition, this tool was based on the Marlowe-Crowne Social Desirability Scale, whose psychometric characteristics have only been evaluated so far within the framework of classical test theory with mixed results. In order to fill the gap in our understanding of the validity of the TALIS social desirability scale within the framework of item response theory, we analyzed the data obtained from a representative sample of Russian teachers. The results showed that the scale had acceptable reliability, significant unidimensionality, and, at the same time, a number of serious problems with its functionality. We propose measures to improve the quality of the psychometric properties of the scale on the basis of the obtained results, including simulated data. We draw fundamental conclusions about the structure of the social desirability construct.