Graph ensemble neural network

Abstract

Ensemble methods have been shown to improve graph neural networks (GNNs). Existing ensemble methods on graphs determine a strong classifier by combining a set of trained base classifiers, i.e., combining the final outputs of base classifiers for prediction. However, these methods fail to promote many popular GNNs to perform well under heterophily (in graphs where many connected nodes have different class labels), which limits their applicability. Furthermore, they ignore the hierarchical nature of GNNs, which results in no interaction between base classifiers when neighbors are aggregated (during training). Two issues arise from this: low applicability and shallow ensemble. We propose Graph Ensemble Neural Network (GEN) for addressing above issues, which is not a simple ensemble of GNNs, but instead integrates ensemble into GNNs to fuse a set of graphs. GEN deepens single ensemble into multiple ensembles during training and applies to homophily and heterophily graphs. In GEN, we design structure augmentation to generate some graphs for training and design feature augmentation for attenuating errors brought by the initial features. Different from existing graph ensemble methods that execute only one ensemble, GEN executes multiple deep ensembles throughout the neighbor aggregation to fuse multiple graphs generated by structure augmentation. Extensive experiments show that GEN achieves new state-of-the-art performance on homophily and heterophily graphs for the semi- and full-supervised node classification. The source code of GEN is publicly available at https://github.com/graphNN/GEN1.