When things go wrong: the recall of AI systems as a last resort for ethical and lawful AI

Abstract

AbstractThis paper presents an initial exploration of the concept of AI system recall, primarily understood as a last resort when AI systems violate ethical norms, societal expectations, or legal obligations. The discussion is spurred by recent incidents involving notable AI systems, demonstrating that AI recalls can be a very real necessity. This study delves into the concept of product recall as traditionally understood in industry and explores its potential application to AI systems. Our analysis of this concept is centered around two prominent categories of recall drivers in the AI domain: ethical-social and legal considerations. In terms of ethical-social drivers, we apply the innovative notion of “moral Operational Design Domain”, suggesting AI systems should be recalled when they violate ethical principles and societal expectation. In addition, we also explore the recall of AI systems from a legal perspective, where the recently proposed AI Act provides regulatory measures for recalling AI systems that pose risks to health, safety, and fundamental rights. The paper also underscores the need for further research, especially around defining precise ethical and societal triggers for AI recalls, creating an efficient recall management framework for organizations, and reassessing the fit of traditional product recall models for AI systems within the AI Act's regulatory context. By probing these complex intersections between AI, ethics, and regulation, this work aims to contribute to the development of robust and responsible AI systems while maintaining readiness for failure scenarios.