Chapter 10 - A systems approach to implementing ethics in a COVID-19 AI application: A qualitative study

Abstract

During the COVID-19 crisis, a range of artificial intelligence (AI) applications have been developed from case identification to population monitoring. The need to develop and deploy AI solutions rapidly because of the urgency of the situation should not come at the expense of ethical considerations or make them an afterthought. Implementing ethics in medical AI is a complex endeavor calling for a systems approach. Such an approach includes a participatory process involving a diverse group of stakeholders. The process needs to be transparent and inclusive and aims to capture the different worldviews at the inception of AI design and development. Using a fictitious scenario based on an aggregation of real COVID-19 apps, we engaged a diverse group of stakeholders such as clinicians, patients, and AI developers in a consultative process. The app included individual self-monitoring of symptoms, physicians’ remote monitoring of symptoms for COVID-19 patients, and tracking of infection clusters by health agencies. Using a systems approach, the participants were tasked with identifying the flow of knowledge between the different agents of the app system and answering questions framing the ethical boundaries of the application, such as identifying the beneficiaries of the app, and the custodians of the process. Because an AI fuel and output are information and knowledge, the types of knowledge exchanged between the different agents of the GP-patient-AI system can shed light on ethical issues. In the context of this study, the content was as important as the process itself and both were the object of the study. Ensuring inclusive, respectful dialogues and candid exchange of points of view is key to establishing a strong foundation for the implementation of ethics and making the process transparent. A systems approach, drawing from Soft Systems Methodology and Critical Systems Heuristics, is well suited to implementing ethics in medical AI.