Algorithmic Decision-Making: Examining the Interplay of People, Technology, and Organizational Practices through an Economic Experiment

Abstract

Human experts are being increasingly required to work with artificial intelligence and machine learning (AI/ML) in organizational decision-making. Using a large-scale historic dataset, we design and run an economic experiment where financially incentivized participants evaluate loan applications with the aid of an AI/ML. We find that humans and AI working together can surpass the AI itself and the humans working alone, under the right conditions. The performance of human-machine teams depends crucially on quality AI technology and well designed organizational practices. Importantly, when both are jointly put into place, firms most significantly increase their profits. We also find that, only when the AI/ML in use has adequate accuracy can the human-machine teams excel humans operating on their own. Otherwise, humans are actually better off working by themselves. We contribute to the emerging algorithmic decision-making literature by examining the properties of both AI/ML technology and organizational policies, in addition to accounting for the human decision makers' characteristics. Importantly, we highlight the importance of their interdependent effect on maximizing organizational outcomes. We especially contribute to the automation literature which investigates which tasks should and should not be automated. Our comparison of human-machine teams vs. machine goes beyond merely pitting human against the machine and is especially important given rising concerns about AI replacing human workers, exacerbating inequality, even eradicating the need for organizational structure. Our findings hold implications for firms wishing to build sustainable human-machine collaboration, that not only serves to increase organizational financial gains, but more importantly, also to understand more clearly the role of humans in the current constantly changing employment landscape due to the rapid advances in AI/ML every day.