Visualizing deep networks using segmentation recognition and interpretation algorithm

Abstract

Owing to the advancements in artificial intelligence (AI) technology, deep learning has found applications in data analysis and problem-related decision-making processes. However, because of the “black-box” characteristics of deep neural networks, the explanations of decision-making processes are ambiguous. Therefore, model developers face challenges while establishing users’ trust in these processes and the associated decision results. Understanding and explaining the decision-making processes implemented by deep learning-based predictive models has importance in various fields.Based on existing interpretation algorithms, this study proposed a novel interpretation algorithm for deep convolutional neural network (CNN) models. The proposed algorithm, known as the neural network interpreter-segmentation recognition and interpretation (NNI-SRI) algorithm, uses the general ideas of segmentation and recognition to interpret models. Compared with other mainstream interpretation algorithms, this algorithm has the following advantages. First, it has a higher model interpretation speed. Second, it can label the positive and negative features observed during the model prediction process. Third, fewer parameters and hardware resources are required because only the forward propagation is used. Furthermore, the proposed algorithm showed better scene adaptability under the premise of ensuring the accuracy of interpretation, which is consistent with the CNN convolution process. Theoretically, the NNI-SRI algorithm can be applied to any model, particularly CNN models, such as InceptionV3, Xception, and ResNet. In this study, the interpretation algorithm was applied to the standard Inception deep network model and the spider sex recognition model developed by us, which achieved excellent results and verified the feasibility of the algorithm.