Interpreting Deep Graph Convolutional Networks with Spectrum Perspective

Abstract

Graph convolutional network (GCN) architecture is the basis of many neural networks and has been widely used in processing graph-structured data. When dealing with large and sparse data, deeper GCN models are often required. However, the models suffer from performance degradation as the number of layers increases. The mainstream attribution of the current research is over-smoothing, and there are also gradient vanishing, training difficulties, etc., so a consensus cannot be reached. In this paper, we theoretically analyze the degradation problem by adopting spectral graph theory to globally consider the propagation and transformation components of the GCN architecture, and conclude that the over-smoothing problem caused by the propagation matrices is not the key factor for performance degradation. Afterwards, in addition to using conventional experimental methods, we proposed an experimental analysis strategy under the guidance of random matrix theory to analyze the singular value distribution of the model weight matrix. We concluded that the key factor leading to the degradation of model performance is the transformation component. In the context of a lack of consensus on the problem of model performance degradation, the paper proposes a systematic analysis strategy, as well as theoretical and empirical evidence.