Understanding community structure in layered neural networks

Abstract

A layered neural network is now one of the most common choices for predicting high-dimensional practical data sets, where the relationship between input and output data is complex and cannot be represented well by simple conventional models. Its effectiveness is shown in various tasks, such as image recognition and natural language processing, however, the lack of interpretability of the trained result by a layered neural network has limited its application area.In our previous studies, we proposed methods for extracting a simplified global structure of a trained layered neural network by applying a network analysis method and by classifying the units into communities according to their connection patterns with adjacent layers. These methods provided us with knowledge about the strength of the relationship between communities from the existence of bundled connections, which are determined by the threshold processing of the connection ratio between pairs of communities.However, it has been difficult to understand the role of each community in detail or quantitatively by observing the resulting modular structure with these previous methods. We could only know to which sets of the input and output dimensions each community was mainly connected by tracing the bundled connections from the community to the input and output layers. Another problem is that the finally obtained modular structure is highly dependent on the setting of the threshold hyperparameter used for determining bundled connections, leading to a different result to the discussion about the role of each community.In this paper, we propose a new method CA-MNN (Community Analysis for Modular Neural Networks), for interpreting quantitatively the role of each community as regards inference, which we extracted using our previously reported methods, by defining the effect of each input dimension on a community, and the effect of a community on each output dimension. We show experimentally that the proposed CA-MNN can reveal the role of each part of a layered neural network by applying the neural networks to three types of data sets, extracting communities from the trained network, and applying the CA-MNN to the community structure.