A graph-based interpretability method for deep neural networks

Abstract

With the development of artificial intelligence, the most representative deep learning has been applied to various fields, which is greatly influencing human society. However, deep neural networks (DNNs) are still a black-box model, and the process how they make decisions internally is still difficult to understand and control. At the same time, DNNs take up more hardware resources, resulting in high energy consumption. Therefore, it is significant to study the characteristics of deep AI models and deeply understand the interactions between parameters within AI models so as to improve the interpretability of DNNs, optimize their structure and increase their computational efficiency. In this paper, we propose a graph-based interpretability method for deep neural networks (GIMDNN). The running parameters of DNNs are modeled as a graph by using a kernel function or the Graph Transformer Networks (GTN), where the nodes of the graph are obtained by dimensional mapping of the parameters of the DNNs, and the edges are calculated by the Gaussian kernel function. The generated graphs are classified by a graph convolutional network (GCN). The association relationship between the adjacent layers and the running mechanism of DNNs are analyzed, and the importance of the parameters of each layer in the DNNs for the final classification result can be obtained. Convolutional neural networks (CNNs) are one of the most representative models in DNNs. The proposed method is experimentally evaluated on the CNNs. The experimental results show that the proposed method can interpret the associations among the weight parameters as well as the correlation between two adjacent layers. Therefore, the DNNs for special tasks, such as portable applications, edge computing, and so on, can be customized, the number of parameters can be reduced. It is valuable to interpret the operation and principle of CNNs.