What is morally at stake when using algorithms to make medical diagnoses? Expanding the discussion beyond risks and harms

Bas De Boer,Olya Kudina

doi:10.1007/s11017-021-09553-0

Abstract

In this paper, we examine the qualitative moral impact of machine learning-based clinical decision support systems in the process of medical diagnosis. To date, discussions about machine learning in this context have focused on problems that can be measured and assessed quantitatively, such as by estimating the extent of potential harm or calculating incurred risks. We maintain that such discussions neglect the qualitative moral impact of these technologies. Drawing on the philosophical approaches of technomoral change and technological mediation theory, which explore the interplay between technologies and morality, we present an analysis of concerns related to the adoption of machine learning-aided medical diagnosis. We analyze anticipated moral issues that machine learning systems pose for different stakeholders, such as bias and opacity in the way that models are trained to produce diagnoses, changes to how health care providers, patients, and developers understand their roles and professions, and challenges to existing forms of medical legislation. Albeit preliminary in nature, the insights offered by the technomoral change and the technological mediation approaches expand and enrich the current discussion about machine learning in diagnostic practices, bringing distinct and currently underexplored areas of concern to the forefront. These insights can contribute to a more encompassing and better informed decision-making process when adapting machine learning techniques to medical diagnosis, while acknowledging the interests of multiple stakeholders and the active role that technologies play in generating, perpetuating, and modifying ethical concerns in health care.

Highlights

Machine learning techniques are increasingly used in medical diagnostic practices, bearing the promise of a more efficient medical diagnosis process with a significantly lower margin of error
We suggested that qualitative moral impacts are underrepresented in the ethical discourse on machine learning (ML), and advanced technomoral change (TMC) and technological mediation theory (TMT) as two approaches that can help to compensate for this underrepresentation
We have shown how the approaches of technomoral change and technological mediation theory can be used to elucidate which qualitative moral impacts are anticipated when ML is introduced into the process of medical diagnosis

Summary

Introduction

Machine learning techniques are increasingly used in medical diagnostic practices, bearing the promise of a more efficient medical diagnosis process with a significantly lower margin of error. The soft impacts of technologies are their qualitative moral effects, such as changes in the extent that doctors can be held responsible for diagnostic outcomes, in the relationship between patients and medical professionals [11], or more generally in the standards for what constitutes good health care practices [12].

Results

Conclusion