Measuring non-cognitive predictors in high-stakes contexts: The effect of self-presentation on self-report instruments used in admission to higher education

Abstract

Non-cognitive constructs such as personality traits and behavioral tendencies show predictive validity for academic performance and incremental validity over and above cognitive constructs. Therefore, non-cognitive predictors are increasingly used in admission procedures for higher education, typically measured using-self-report instruments. It is well known that self-report instruments are sensitive to self-presentation, especially in high-stakes contexts. However, remarkably few studies investigated the effect of self-presentation on predictive validity. The effect of self-presentation in applicants to an undergraduate psychology program was studied using a repeated measures design. Respondents completed self-report questionnaires measuring non-cognitive predictors of academic performance before admission to the program, and again after admission. Scores were compared between contexts, as well as predictive validity, incremental validity, and potential hiring decisions. Results showed differences in scores between contexts on all scales, attenuated predictive validity for most scales, attenuated incremental validity when scores obtained in the admission context were used, and effects on admission decisions. In conclusion, validity results based on scores measured in low-stakes contexts cannot simply be generalized to high-stakes contexts. Furthermore, results obtained in a high-stakes context may result in self-presentation irrespective of whether participants are informed that their scores are used for selection decisions or not.