Justificatory explanations: a step beyond explainability in machine learning

Abstract

Abstract AI systems may have many potential negative effects, so understanding how they generate results is important. AI explainability is a crucial but highly technical field that might be inaccessible to many experts in (public) health. We present a non-technical approach to the issue and is focused on the conceptual foundations for decisions made by developers and stakeholders during the system's lifecycle. To illustrate, imagine an app that tracks snoring to offer health recommendations. A classical explainability approach would be to evaluate the quality of the training dataset and the model itself. We argue that if we want to understand the system, we also need to look at the notion of “health” as it is its basic conceptual underpinning. We believe that these conceptual foundations as well as rich descriptions of goals and purposes that are pursued can be integrated into what we call ‘justificatory explanations'. These would illustrate how key concepts influence design and development decisions offering thus valuable insights into the workings and outcomes of the explained system. ‘Justificatory explanations’ are declarative statements written in plain language that provide a high-level account of the team's understanding of the relevant key concepts related to the project's main domain and how these understandings drive decision-making during the lifecycle stages. We propose that ‘justificatory explanations’ should be incorporated into the technical documentation that usually accompanies the release or deployment of a system. In short, ‘justificatory explanations’ complement other efforts around explainability by highlighting the reasons the person or persons designing, developing, selling and deploying the system consider to have plausible justificatory power for the decisions that were made during the project. This increased conceptual awareness may be beneficial to all, even to those without a background in mathematics, and data and computer science. Key messages • Explainability in AI should go beyond the data and the model to integrate design decisions. • Justificatory reasons about the system should be provided in plain language.