Building truths in AI: Making predictive algorithms doable in healthcare

Abstract

ABSTRACT Increasingly, artificial intelligent (AI) algorithms are being applied to automatically assist or automate decisions. Such statistical models have been criticized in the existing literature especially for producing cultural biases and for challenging our notions of knowledge. However, few studies have contributed to an essential understanding of the way in which algorithms are designed with particular truths to enable systematic decision-making. Drawing on an ethnographic study in a Scandinavian AI company, this article analyzes how truth is built through layered interpretative practices in applied AI for healthcare, and critically assesses how such practices shed light on the pragmatic notion of truth(s) in AI. The study identifies five practices that all show difficulty in modeling fuzzy patient conditions into one firm truth. The key contribution is that truth goes from being a process of discovering a more ‘right’ truth to become a process of reinventing the existing truth and healthcare practice. These findings suggest that truth in applied AI is a key devise for making predictive algorithms a viable business, and that developers are in a favorable position to make not only AI doable but also the very truth they intend to find and model. The study in this way shows how change is an inherent part of making AI systems, and that centralizing truth practices is a fruitful way of analyzing such changes and developers’ agency. We argue for analytical awareness of how AI truth practices may prompt a world that is fit to algorithms rather than a world to which algorithms are fit.