AI Transparency in a Real-World Context: What we can learn from past examples of algorithmic and statistical decision-making

Abstract

Public discussion about transparency for AI-enabled decisions tends to focus on the challenge of AI explainability. However, there are additional real-world factors which can hamper individuals seeking to understand or challenge decisions impacting them, even when the AI or algorithm is entirely explainable. Although AI enabled decision tools are relatively new, algorithmic and statistical decision tools are not. This paper examines past efforts by individuals to access algorithms, statistical models, and data used in making decisions which impacted them. The results of those attempts are considered in light of public expectations for transparency of AIenabled decision tools, as well as current and developing guidance. Legal changes will be needed if governments wish to meet citizen expectations for real-world transparency of AI-enabled decision systems. In the meantime, there are opportunities for AI experts and others to protect the potential for greater transparency through open data, open source licensing, and engagement in policy development