Scoring a forced-choice image-based assessment of personality: A comparison of machine learning, regression, and summative approaches

Abstract

Recent years have seen rapid advancements in the way that personality is measured, resulting in a number of innovative predictive measures being proposed, including using features extracted from videos and social media profiles. In the context of selection, game- and image-based assessments of personality are emerging, which can overcome issues like social desirability bias, lack of engagement and low response rates that are associated with traditional self-report measures. Forced-choice formats, where respondents are asked to rank responses, can also mitigate issues such as acquiescence and social desirability bias. Previously, we reported on the development of a gamified forced-choice image-based assessment of the Big Five personality traits created for use in selection, using Lasso regression for the scoring algorithms. In this study, we compare the machine-learning-based Lasso approach to ordinary least squares regression, as well as the summative approach that is typical of forced-choice formats. We find that the Lasso approach performs best in terms of generalisability and convergent validity, although the other methods have greater discriminate validity. We recommend the use of predictive Lasso regression models for scoring forced-choice image-based measures of personality over the other approaches. Potential further studies are suggested.