Predicting decision-makers’ algorithm use

Abstract

Decision makers typically integrate multiple pieces of information to make predictions and decisions. They also sometimes receive algorithmic advice, but often discount such advice. This usually results in less consistent and less accurate predictions than consistently using the advice. We hypothesized that individual differences on psychological traits such as dutifulness (a facet of conscientiousness), decision-making styles, and predictor validity beliefs are related to the consistent use of algorithmic advice, judgment consistency, and predictive validity. We sampled participants with hiring experience (N = 308) who predicted the performance of job applicants based on test scores and interview ratings, and they also received algorithmic advice. The results showed that more dutiful participants and participants with more accurate predictor validity beliefs used the algorithm more, and made more consistent and more accurate predictions. We did not find evidence that an intuitive decision-making style was related to these outcomes. Exploratory analyses showed that cognitive ability was positively related to the consistent use of algorithmic advice and judgment consistency, but not significantly related to predictive validity. Furthermore, the other conscientiousness facets and the general factor were similarly related to the outcome variables as dutifulness. Organizations may want to hire conscientious decision makers, and decision makers with accurate predictor validity beliefs. In addition, organizations could provide training on predictor validities.