Guiding supervisors in artificial intelligence-enabled forecasting: Understanding the impacts of salience and detail on decision-making

Abstract

In many real-world situations, multiple humans are involved in decision-making when interacting with machine recommendations. We investigated a setting where an artificial intelligence system creates demand forecasts that a human planner can either accept or revise, and a supervisor then makes the final decision about which forecast to select. We designed and conducted two experimental studies to understand decision-making by a supervisor. First, we provided the improvement probabilities of adjustments at an aggregated level and found evidence for overoptimism bias and mean anchoring. Second, we provided decomposed guidance based on two adjustment attributes, direction and magnitude, to investigate the role of salience based on the distance between the improvement probabilities and level of detail in guidance effectiveness. We found no significant difference in using less and more salient guidance provided that the detail level was fixed. However, revealing more details when the guidance was more salient increased the use of guidance.