Holistic Graph Neural Networks based on a global-based attention mechanism

Abstract

Graph Neural Networks (GNNs) have become increasingly popular due to their impressive capacity to perform classification or regression on high-dimensional graph-structured data. However, standard message passing GNNs typically define nodes embeddings through a recursive neighborhood aggregation process which updates the representation vector of each node with reference to its neighborhood only. In this paper, we propose the Holistic Graph Neural Network (HGNN), a two-fold architecture which introduces a global-based attention mechanism for learning and generating nodes embeddings. The global features we inject, summarize the overall global behavior of the graph in addition to the local semantic and structural information. These global features will make each individual node aware of the global behavior of the graph outside the borders of the local neighborhood. We further propose a variant of the HGNN, we call HGNNα based on a more sophisticated hierarchical global-feature extraction mechanism. We explore diverse global pooling strategies to derive highly expressive global features. We also show that state-of-the-art GNNs can significantly benefit from the addition of the global-based attention introduced. Furthermore, we prove the efficiency of the HGNN model theoretically and adapt it to support graph data which carries edge attributes for example the Molecular datasets from the Open Graph Benchmark. Experiments on Bioinformatics datasets, Social Networks and Molecular datasets demonstrate that our proposed models achieve much better performance than state-of-the-art methods, for instance we achieved improvements of +11% on COLLAB and +13% on IMDB-BINARY datasets.