BI-CAM: Generating Explanations for Deep Neural Networks Using Bipolar Information

Abstract

The higher requirements for deep neural networks are driving researchers to have a deeper understanding of the internals of neural networks. The class activation map (CAM) based methods can provide a convincing interpretation of the features extracted by the neural network from both visual and quantitative perspectives. However, the existing CAM methods do not take into account that the non-target region also contains target-related activation, which results in the generated saliency map containing noise from unrelated regions. In addition, the soft mask with continuous value not only contains more non-target regions for gradient-free CAM, but also causes the characteristics and distribution of the target region to be disturbed. This paper proposed a novel CAM method named Bipolar Information CAM (BI-CAM) to interpret convolutional neural networks (CNNs) and graph convolutional networks (GCNs). Firstly, dual-stream information is proposed to precisely quantify the relationship between the target region and the non-target region for an image/graph. Secondly, binary reformation is also proposed to generate a hard mask that can retain the original features and regions. Finally, we propose to use concise and effective Point-wise Mutual Information (PMI) to measure the quantitative relationship between the image and the local region with respect to the label. The results of the experiment show that the proposed BI-CAM achieves significantly better performance in the faithfulness evaluation from the perspectives of visualization and quantitative analysis than other competitive interpretation methods.