SIGNed explanations: Unveiling relevant features by reducing bias

Abstract

The rise of artificial intelligence (AI) is accompanied by a growing need for methods to explain the decisions of AI models. In the last decade, new explanation methods have been developed in the field of explainable AI (XAI) that enable the interpretation of predictions and provide additional information to human experts. In computer vision tasks, XAI methods are typically used to compute heatmaps to identify regions or pixels of an image that were particularly relevant to the output of the AI model. Heatmaps computed on deep neural networks often appear visually noisy due to the Shattered Gradients Problem. Therefore, several explanation methods use multiplication by the input values to increase contrast and reduce noise in heatmaps. However, this can induce a bias with the magnitude of the input values in the explanation. In this work, we proposed a novel technique called SIGN, which can be combined with existing XAI methods to prevent this bias. We implemented SIGN-based variants of the existing XAI methods Gradient × Input, SmoothGrad, DeconvNet, Guided Backpropagation, and Layer-wise Relevance Propagation (LRP), called Gradient × SIGN, SmoothGrad × SIGN, DeconvNet × SIGN, Guided Backpropagation × SIGN, and LRP-SIGN, and compared them to ten state-of-the-art XAI methods on MNIST, ImageNet, and MIT Places365 using different dense and convolutional model architectures. The SIGNed explanations were able to outperform the existing methods in all tested metrics. The code to reproduce the experiments is available from GitHub via https://github.com/nilsgumpfer/SIGN.