The generalizability of machine learning models of personality across two text domains

Abstract

Machine learning of high-dimensional models have received attention for their ability to predict psychological variables, such as personality. However, it has been less examined to what degree such models are capable of generalizing across domains. Across two text domains (Reddit message and personal essays), compared to low-dimensional- and theoretical models, atheoretical high-dimensional models provided superior predictive accuracy within but poor/non-significant predictive accuracy across domains. Thus, complex models depended more on the specifics of the trained domain. Further, when examining predictors of models, few survived across domains. We argue that theory remains important when conducting prediction-focused studies and that research on both high- and low-dimensional models benefit from establishing conditions under which they generalize.