Graph Ensemble Networks for Semi-supervised Embedding Learning

Abstract

Recently, semi-supervised graph learning has attracted growing research interests. Since the Graph Convolutional Network (GCN) was formulated, some studies argue that shallow architectures fall through over-fitting and have limited ability to aggregate information from high-order neighbors. However, although deep GCNs have powerful nonlinear fitting ability, problems such as over-smoothing still exist with the expansion of layer depth. In this paper, we explore such an ignored question that whether a shallow GCN can achieve significant improvement over deep GCNs. Motivated by this curiosity, we propose an effective graph learning framework—Graph Ensemble Network (GENet) for semi-supervised learning tasks. We combine ensemble learning with data augmentation, which samples multi-subgraphs by randomly removing some edges to generate different topology spaces, and knowledge distillation is introduced to make the multi-subnetworks learn collaboratively. The central idea is that employ attention mechanism and consistency constraint to learn adaptive importance weights of the embeddings from the ensemble model, and then transfer learned knowledge to a shallow GCN. Extensive experiments on graph node classification verify the superiority of the proposed GENet compared with the state-of-the-art methods.