Perception is reality? Understanding user perceptions of chatbot-inferred versus self-reported personality traits

Abstract

Artificial Intelligence (AI) can infer one's personality from online behavior, which offers an interesting alternative to traditional, self-reported personality assessments. Recent studies comparing AI-inferred personality to personality derived from traditional assessments have found noticeable differences between the two (meta-analyses have found mean correlations of 0.3 between AI-inferred personality and personality from surveys). One important but unanswered question is how users perceive their personality derived from both methods. Which do users perceive to be more accurate, and more satisfying to use? To answer this question, we used both methods to conduct personality assessments of 595 participants and then asked users how well the two sets of results fit them, as well as their satisfaction and intention to use them. Participants reported that both results fit them equally well, even though the two methods reported different personality scores. Users were equally satisfied with both methods but were more likely to use the survey, likely because it took less time. Our findings imply that both methods measure different aspects of user personality, and both may be useful. We discuss the pros and cons of AI-inferred versus traditional, self-reported personality and indicate future research directions of AI-inferred personality assessment and the implications of their use for real-world applications.