On the failings of Shapley values for explainability

Abstract

Explainable Artificial Intelligence (XAI) is widely considered to be critical for building trust into the deployment of systems that integrate the use of machine learning (ML) models. For more than two decades Shapley values have been used as the theoretical underpinning for some methods of XAI, being commonly referred to as SHAP scores. Some of these methods of XAI now rank among the most widely used, including in high-risk domains. This paper proves that the existing definitions of SHAP scores will necessarily yield misleading information about the relative importance of features for predictions. The paper identifies a number of ways in which misleading information can be conveyed to human decision makers, and proves that there exist classifiers which will yield such misleading information. Furthermore, the paper offers empirical evidence that such theoretical limitations of SHAP scores are routinely observed in ML classifiers.