“Inventor’s Bias” at Work: When Low-Performing Algorithms Seem Fair

Abstract

This article introduces the “Inventor’s Bias Effect,” the propensity for inventors to be over-optimistic about the positive features and uses of the products they create. We explore this phenomenon in the context of decision-making algorithms by conducting two online studies (N = 1001) where subjects were asked to either create or evaluate an AI-based tool that can automate human resource decisions in an organization. Study 1 revealed that individuals in the role of inventor perceived a low-performing algorithm they created as fairer relative to the ratings of other stakeholders (CEOs, employees, and the general public). The tendency for these “inventors” to personally identify with the products they created mediated this effect. Study 2 showed that inventors’ perceptions of fairness of the algorithms they created translated into an increased desire for the organization to continue using their product, even though it was inaccurate for a third of all decisions. This research demonstrates how stakeholders’ relations to algorithms may encourage biased decision making and highlights the need for caution in organizational and political decision-making processes.