Governing Ethical Gaps in Distributed AI Development

Abstract

Good business practice often leads companies to subdivide into separate functional entities for operational efficiency and specialization. However, these kinds of divisions can generate significant ethical and perhaps even regulatory gaps when they occur in AI companies. In particular, one natural division for an AI company is into separate entities responsible for model development, testing, and cybersecurity (to maintain and protect data). In this paper, we argue that this division can lead to some ethical responsibilities always being “someone else’s job.” For concreteness, we consider the US National Institute of Standards and Technology’s AI Risk Management Framework (NIST AI RMF) as a guide to ethical obligations in a corporate context. We show that a common division of labor in AI development and deployment can lead to specific obligations for which no entity is responsible, even though they apply to the effort as a whole. We propose “Join Accountability Agreements”, a mechanism to ensure that ethical obligations do not slip through the cracks because of the way an effort is structured. We thus aim to highlight the significance of comprehensive examinations of and adaptable strategies for our ethical obligations when developing AI systems in a distributed manner.