Gated Graph Neural Network Research Articles

GGAS2SN: Gated Graph and SmilesToSeq Network for Solubility Prediction.

Aqueous solubility is a critical physicochemical property of drug discovery. Solubility is a key issue in pharmaceutical development because it can limit a drug's absorption capacity. Accurate solubility prediction is crucial for pharmacological, environmental, and drug development studies. This research introduces a novel method for solubility prediction by combining gated graph neural networks (GGNNs) and graph attention neural networks (GATs) with Smiles2Seq encoding. Our methodology involves converting chemical compounds into graph structures with nodes representing atoms and edges indicating chemical bonds. These graphs are then processed by using a specialized graph neural network (GNN) architecture. Incorporating attention mechanisms into GNN allows for capturing subtle structural dependencies, fostering improved solubility predictions. Furthermore, we utilized the Smiles2Seq encoding technique to bridge the semantic gap between molecular structures and their textual representations. Smiles2Seq seamlessly converts chemical notations into numeric sequences, facilitating the efficient transfer of information into our model. We demonstrate the efficacy of our approach through comprehensive experiments on benchmark solubility data sets, showcasing superior predictive performance compared to traditional methods. Our model outperforms existing solubility prediction models and provides interpretable insights into the molecular features driving solubility behavior. This research signifies an important advancement in solubility prediction, offering potent tools for drug discovery, formulation development, and environmental assessments. The fusion of GGNN and Smiles2Seq encoding establishes a robust framework for accurately forecasting solubility across various chemical compounds, fostering innovation in various domains reliant on solubility data.

Gated graph spiking neural P network for session-based recommendation

Session-based recommendation (SBR) is a recommendation system application scenario that focuses on user interests and behaviors in sessions formed within a short period. However, existing SBR methods do not fully extract user contextual information, and gated graph neural networks(GNNs) still face challenges in handling long-term dependencies. The nonlinear spiking neural P system is a novel spiking neural-like computing model in which the nonlinear spiking mechanism is a typical feature. Based on this nonlinear spiking mechanism, we propose a new recurrent-like model called the extended gated spiking neural P model or the EGSNP model, which can effectively capture temporal relationships in their sequence and obtain the user’s contextual information to alleviate information extraction problems in SBR. Based on this EGSNP model, we propose a new type of GNN called the extended gated graph spiking neural P network or termed the EGGSNP network. Finally, we use the EGSNP model and EGGSNP network to construct a new SBR model for the session recommendation task, termed the SR-EGGSNP model. This SR-EGGSNP model can effectively extract contextual information and flexibly capture the long-term evolution of interest while alleviating the long-distance dependency problem in GNNs. We conducted extensive comparative experiments on two public datasets, and the results demonstrated the effectiveness of the proposed model and accuracy of the recommendations.

GraphPyRec: A novel graph-based approach for fine-grained Python code recommendation

Artificial intelligence has been widely applied in software engineering areas such as code recommendation. Significant progress has been made in code recommendation for static languages in recent years, but it remains challenging for dynamic languages like Python as accurately determining data flows before runtime is difficult. This limitation hinders data flow analysis, affecting the performance of code recommendation methods that rely on code analysis. In this study, a graph-based Python recommendation approach (GraphPyRec) is proposed by converting source code into a graph representation that captures both semantic and dynamic information. Nodes represent semantic information, with unique rules defined for various code statements. Edges depict control flow and data flow, utilizing a child-sibling-like process and a dedicated algorithm for data transfer extraction. Alongside the graph, a bag of words is created to include essential names, and a pre-trained BERT model transforms it into vectors. These vectors are integrated into a Gated Graph Neural Network (GGNN) process of the code recommendation model, enhancing its effectiveness and accuracy. To validate the proposed method, we crawled over a million lines of code from GitHub. Experimental results show that GraphPyRec outperforms existing mainstream Python code recommendation methods, achieving Top-1, 5, and 10 accuracy rates of 68.52%, 88.92%, and 94.05%, respectively, along with a Mean Reciprocal Rank (MRR) of 0.772.

Function-Level Compilation Provenance Identification with Multi-Faceted Neural Feature Distillation and Fusion

In the landscape of software development, the selection of compilation tools and settings plays a pivotal role in the creation of executable binaries. This diversity, while beneficial, introduces significant challenges for reverse engineers and security analysts in deciphering the compilation provenance of binary code. To this end, we present MulCPI, short for Multi-representation Fusion-based Compilation Provenance Identification, which integrates the features collected from multiple distinct intermediate representations of the binary code for better discernment of the fine-grained function-level compilation details. In particular, we devise a novel graph-oriented neural encoder improved upon the gated graph neural network by subtly introducing an attention mechanism into the neighborhood nodes’ information aggregation computation, in order to better distill the more informative features from the attributed control flow graph. By further integrating the features collected from the normalized assembly sequence with an advanced Transformer encoder, MulCPI is capable of capturing a more comprehensive set of features manifesting the multi-faceted lexical, syntactic, and structural insights of the binary code. Extensive evaluation on a public dataset comprising 854,858 unique functions demonstrates that MulCPI exceeds the performance of current leading methods in identifying the compiler family, optimization level, compiler version, and the combination of compilation settings. It achieves average accuracy rates of 99.3%, 96.4%, 90.7%, and 85.3% on these tasks, respectively. Additionally, an ablation study highlights the significance of MulCPI’s core designs, validating the efficiency of the proposed attention-enhanced gated graph neural network encoder and the advantages of incorporating multiple code representations.

A heterogeneous traffic spatio-temporal graph convolution model for traffic prediction

Smart cities require advanced traffic management systems. Traffic forecasting is an essential task of the advanced transportation system. Traffic spatio-temporal data are often heterogeneous. Most existing traffic prediction models predominantly use separate components to extract the temporal and spatial features of traffic data. However, this overlooks the intrinsic connections between the spatio-temporal features of traffic data. To directly mine the spatio-temporal heterogeneity, this study constructs a global heterogeneous traffic spatio-temporal graph and proposes the Heterogeneous Traffic Spatio-Temporal Graph Convolution (HTSTGC). To reduce the complexity of the model, Simple Graph Convolution (SGC) is used to extract semi-structured meta-graph information. The receptive fields that capture temporal and spatial features can be flexibly adjusted separately through clever design, which can balance the performance and efficiency of the model. Finally, the feature fusion module applies Gated Graph Neural Network (GGNN) to fuse temporal and spatial features. The results on the PEMS datasets reveal that jointly modeling different types of relationships can improve the traffic prediction performance of the model. The proposed HTSTGC has better performance than the baseline methods in most cases. The research results can support urban traffic control, traffic pollution reduction, and sustainable urban development.

Document-Level Relation Extraction with Deep Gated Graph Reasoning

Extracting the relations of two entities on the sentence-level has drawn increasing attention in recent years but remains facing great challenges on document-level, due to the inherent difficulty in recognizing the relations of two entities across multiple sentences. Previous works show that employing the graph convolutional neural network can help the model capture unstructured dependent information of entities. However, they usually employed the non-adaptive weight edges to build the correlation weight matrix which suffered from the problem of information redundancy and gradient disappearance. To solve this problem, we propose a deep gated graph reasoning model for document-level relation extraction, namely, BERT-GGNNs, which employ an improved gated graph neural network with a learnable correlation weight matrix to establish multiple deep gated graph reason layers. The proposed deep gated graph reasoning layers make the model easier to reasoning the relations between entities hidden in the document. Experiments show that the proposed model outperforms most of strong baseline models, and our proposed model is 0.3% and 0.3% higher than the famous LSR-BERT model on the F1 and Ing F1, respectively.

Prerequisite-Enhanced Category-Aware Graph Neural Networks for Course Recommendation

The rapid development of Massive Open Online Courses (MOOCs) platforms has created an urgent need for an efficient personalized course recommender system that can assist learners of all backgrounds and levels of knowledge in selecting appropriate courses. Currently, most existing methods utilize a sequential recommendation paradigm that captures the user’s learning interests from their learning history, typically through recurrent or graph neural networks. However, fewer studies have explored how to incorporate principles of human learning at both the course and category levels to enhance course recommendations. In this article, we aim at addressing this gap by introducing a novel model, named Prerequisite-Enhanced Catory-Aware Graph Neural Network (PCGNN), for course recommendation. Specifically, we first construct a course prerequisite graph that reflects the human learning principles and further pre-train the course prerequisite relationships as the base embeddings for courses and categories. Then, to capture the user’s complex learning patterns, we build an item graph and a category graph from the user’s historical learning records, respectively: (1) the item graph reflects the course-level local learning transition patterns and (2) the category graph provides insight into the user’s long-term learning interest. Correspondingly, we propose a user interest encoder that employs a gated graph neural network to learn the course-level user interest embedding and design a category transition pattern encoder that utilizes GRU to yield the category-level user interest embedding. Finally, the two fine-grained user interest embeddings are fused to achieve precise course prediction. Extensive experiments on two real-world datasets demonstrate the effectiveness of PCGNN compared with other state-of-the-art methods.

Infusing external knowledge into user stance detection in social platforms

Stance detection for user reviews on social platforms aims to classify the stance of users’ reviews toward a specific topic. Existing studies focused on the internal semantic features of reviews’ texts, but ignored the external knowledge associated with the review. This paper retrieves external knowledge related to the key information of each review by mapping it to a knowledge graph. Thereafter, this paper infuses the external knowledge into deep learning model for stance detection. Considering that infusing external knowledge may bring noise to the model, this paper adopts the personalized PageRank method to filter the introduced irrelevant external knowledge. Infusing external knowledge can improve the classification performance by providing background knowledge. In addition to considering the textual features of reviews when constructing the stance detection model, this paper employs a gated graph neural network (GGNN) approach to fuse the structural information between reviews to capture the interactions of reviews. The experiments show that the model improves 1.5% –6.9% in macro-average scores compared to six benchmark models in this paper. By combining the textual features and structural information of reviews and introducing external knowledge, the model effectively improves the stance detection performance.

SolPredictor: Predicting Solubility with Residual Gated Graph Neural Network.

Computational methods play a pivotal role in the pursuit of efficient drug discovery, enabling the rapid assessment of compound properties before costly and time-consuming laboratory experiments. With the advent of technology and large data availability, machine and deep learning methods have proven efficient in predicting molecular solubility. High-precision in silico solubility prediction has revolutionized drug development by enhancing formulation design, guiding lead optimization, and predicting pharmacokinetic parameters. These benefits result in considerable cost and time savings, resulting in a more efficient and shortened drug development process. The proposed SolPredictor is designed with the aim of developing a computational model for solubility prediction. The model is based on residual graph neural network convolution (RGNN). The RGNNs were designed to capture long-range dependencies in graph-structured data. Residual connections enable information to be utilized over various layers, allowing the model to capture and preserve essential features and patterns scattered throughout the network. The two largest datasets available to date are compiled, and the model uses a simplified molecular-input line-entry system (SMILES) representation. SolPredictor uses the ten-fold split cross-validation Pearson correlation coefficient R2 0.79±0.02 and root mean square error (RMSE) 1.03±0.04. The proposed model was evaluated using five independent datasets. Error analysis, hyperparameter optimization analysis, and model explainability were used to determine the molecular features that were most valuable for prediction.

Classification of epileptic seizures in EEG data based on iterative gated graph convolution network.

The automatic and precise classification of epilepsy types using electroencephalogram (EEG) data promises significant advancements in diagnosing patients with epilepsy. However, the intricate interplay among multiple electrode signals in EEG data poses challenges. Recently, Graph Convolutional Neural Networks (GCN) have shown strength in analyzing EEG data due to their capability to describe complex relationships among different EEG regions. Nevertheless, several challenges remain: (1) GCN typically rely on predefined or prior graph topologies, which may not accurately reflect the complex correlations between brain regions. (2) GCN struggle to capture the long-temporal dependencies inherent in EEG signals, limiting their ability to effectively extract temporal features. To address these challenges, we propose an innovative epileptic seizure classification model based on an Iterative Gated Graph Convolutional Network (IGGCN). For the epileptic seizure classification task, the original EEG graph structure is iteratively optimized using a multi-head attention mechanism during training, rather than relying on a static, predefined prior graph. We introduce Gated Graph Neural Networks (GGNN) to enhance the model's capacity to capture long-term dependencies in EEG series between brain regions. Additionally, Focal Loss is employed to alleviate the imbalance caused by the scarcity of epileptic EEG data. Our model was evaluated on the Temple University Hospital EEG Seizure Corpus (TUSZ) for classifying four types of epileptic seizures. The results are outstanding, achieving an average F1 score of 91.5% and an average Recall of 91.8%, showing a substantial improvement over current state-of-the-art models. Ablation experiments verified the efficacy of iterative graph optimization and gated graph convolution. The optimized graph structure significantly differs from the predefined EEG topology. Gated graph convolutions demonstrate superior performance in capturing the long-term dependencies in EEG series. Additionally, Focal Loss outperforms other commonly used loss functions in the TUSZ classification task.

SAGN: Sparse Adaptive Gated Graph Neural Network With Graph Regularization for Identifying Dual-View Brain Networks.

Due to the absence of a gold standard for threshold selection, brain networks constructed with inappropriate thresholds risk topological degradation or contain noise connections. Therefore, graph neural networks (GNNs) exhibit weak robustness and overfitting problems when identifying brain networks. Furthermore, existing studies have predominantly focused on strongly coupled connections, neglecting substantial evidence from other intricate systems that highlight the value of weakly coupled connections. Consequently, the potential of weakly coupled brain networks remains untapped. In this study, we pioneeringly construct weakly coupled brain networks and validate their values in emotion identification tasks. Subsequently, we propose a sparse adaptive gated GNN (SAGN) that can simultaneously perceive the valuable topology of dual-view (i.e., strongly coupled and weakly coupled) brain networks. The SAGN contains a sparse adaptive global receptive field. Moreover, SAGN employs a gated mechanism with feature enhancement and adaptive noise suppression capabilities. To address the lack of inductive bias and the large capacity of SAGN, a graph regularization term built with prior topology of dual-view brain networks is introduced to enhance generalization. Besides a public dataset (SEED), we also built a custom dataset (MuSer) with 60 subjects to evaluate weakly coupled brain networks' value and validate the SAGN's performance. Experiments demonstrate that brain physiological patterns associated with different emotional states are separable and rooted in weakly coupled brain networks. In addition, SAGN exhibits excellent generalization and robustness in identifying brain networks.

Input State Stability of Gated Graph Neural Networks

Input State Stability of Gated Graph Neural Networks

A unified framework for backpropagation-free soft and hard gated graph neural networks

We propose a framework for the definition of neural models for graphs that do not rely on backpropagation for training, thus making learning more biologically plausible and amenable to parallel implementation. Our proposed framework is inspired by Gated Linear Networks and allows the adoption of multiple graph convolutions. Specifically, each neuron is defined as a set of graph convolution filters (weight vectors) and a gating mechanism that, given a node and its topological context, generates the weight vector to use for processing the node’s attributes. Two different graph processing schemes are studied, i.e., a message-passing aggregation scheme where the gating mechanism is embedded directly into the graph convolution, and a multi-resolution one where neighboring nodes at different topological distances are jointly processed by a single graph convolution layer. We also compare the effectiveness of different alternatives for defining the context function of a node, i.e., based on hyperplanes or on prototypes, and using a soft or hard-gating mechanism. We propose a unified theoretical framework allowing us to theoretically characterize the proposed models’ expressiveness. We experimentally evaluate our backpropagation-free graph convolutional neural models on commonly adopted node classification datasets and show competitive performances compared to the backpropagation-based counterparts.

Gated graph neural networks for identifying contamination sources in water distribution systems

Contamination events in water distribution networks (WDN) pose significant threats to water supply and public health. Rapid and accurate contamination source identification (CSI) can facilitate the development of remedial measures to reduce impacts. Though many machine learning (ML) methods have been proposed for fast detection, there is a critical need for approaches capturing complex spatial dynamics in WDNs to enhance prediction accuracy. This study proposes a gated graph neural network (GGNN) for CSI in the WDN, incorporating both spatiotemporal water quality data and flow directionality between network nodes. Evaluated across various contamination scenarios, the GGNN demonstrates high prediction accuracy even with limited sensor coverage. Notably, directional connections significantly enhance the GGNN CSI accuracy, underscoring the importance of network topology and flow dynamics in ML-based WDN CSI approaches. Specifically, the method achieves a 92.27% accuracy in narrowing the contamination source to 5 points using just 2 h of sensor data. The GGNN showcases resilience under model and measurement uncertainties, reaffirming its potential for real-time implementation in practice. Moreover, our findings highlight the impact of sensor sampling frequency and measurement accuracy on CSI accuracy, offering practical insights for ML methods in water network applications.

Real-time water quality prediction in water distribution networks using graph neural networks with sparse monitoring data

Ensuring the safety and reliability of drinking water supply requires accurate prediction of water quality in water distribution networks (WDNs). However, existing hydraulic model-based approaches for system state prediction face challenges in model calibration with limited sensor data and intensive computing requirements, while current machine learning models are lack of capacity to predict the system states at sites that are not monitored or included in model training. To address these gaps, this study proposes a novel gated graph neural network (GGNN) model for real-time water quality prediction in WDNs. The GGNN model integrates hydraulic flow directions and water quality data to represent the topology and system dynamics, and employs a masking operation for training to enhance prediction accuracy. Evaluation results from a real-world WDN demonstrate that the GGNN model is capable to achieve accurate water quality prediction across the entire WDN. Despite being trained with water quality data from a limited number of sensor sites, the model can achieve high predictive accuracies (Mean Absolute Error = 0.07 mg L−1 and Mean Absolute Percentage Error = 10.0 %) across the entire network including those unmonitored sites. Furthermore, water quality-based sensor placement significantly improves predictive accuracy, emphasizing the importance of careful sensor location selection. This research advances water quality prediction in WDNs by offering a practical and effective machine learning solution to address challenges related to limited sensor data and network complexity. This study provides a first step towards developing machine learning models to replace hydraulic models in WDN modelling.

Tensor‐based gated graph neural network for automatic vulnerability detection in source code

AbstractThe rapid expansion of smart devices leads to the increasing demand for vulnerability detection in the cyber security field. Writing secure source codes is crucial to protect applications and software. Recent vulnerability detection methods are mainly using machine learning and deep learning. However, there are still some challenges, how to learn accurate source code semantic embedding at the function level, how to effectively perform vulnerability detection using the learned semantic embedding of source code and how to solve the overfitting problem of learning‐based models. In this paper, we consider codes as various graphs with node features and propose a tensor‐based gated graph neural network called TensorGNN to produce code embedding for function‐level vulnerability detection. First, we propose a high‐dimensional tensor for combining different code graph representations. Second, inspired by the work of tensor technology, we propose the TensorGNN model to produce accurate code representations using the graph tensor. We evaluate our model on 7 C and C++ large open‐source code corpus (e.g. SARD&NVD, Debian, SATE IV, FFmpeg, libpng&LibTiff, Wireshark and Github datasets), which contains 13 types of vulnerabilities. Our TensorGNN model improves on existing state‐of‐the‐art works by 10%–30% on average in terms of vulnerability detection accuracy and F1, while our TensorGNN model needs less training time and model parameters. Specifically, compared with other existing works, our model reduces 25–47 times of the number of parameters and decreases 3–10 times of training time. Results of evaluations show that TensorGNN has better performance while using fewer training parameters and less training time.

Enhancing Traffic Density Detection and Synthesis through Topological Attributes and Generative Methods

This study investigates the utilization of Graph Neural Networks (GNNs) within the realm of traffic forecasting, a critical aspect of intelligent transportation systems. The accuracy of traffic predictions is pivotal for various applications, including trip planning, road traffic control, and vehicle routing. The research comprehensively explores three notable GNN architectures—Graph Convolutional Networks (GCNs), GraphSAGE (Graph Sample and Aggregation), and Gated Graph Neural Networks (GGNNs)—specifically in the context of traffic prediction. Each architecture's methodology is meticulously examined, encompassing layer configurations, activation functions, and hyperparameters. With the primary aim of minimizing prediction errors, the study identifies GGNNs as the most effective choice among the three models. The outcomes, presented in terms of Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE), reveal intriguing insights. While GCNs exhibit an RMSE of 9.25 and an MAE of 8.2, GraphSAGE demonstrates improved performance with an RMSE of 8.5 and an MAE of 7.6. Gated Graph Neural Networks (GGNNs) emerge as the leading model, showcasing the lowest RMSE of 9.2 and an impressive MAE of 7.0. However, the study acknowledges the dynamic nature of these results, emphasizing their dependency on factors such as the dataset, graph structure, feature engineering, and hyperparameter tuning.

SENSE: An unsupervised semantic learning model for cross-platform vulnerability search

Binary Similarity Analysis (BSA) emerges as a vital approach for identifying homologous vulnerabilities. However, it is constrained by semantic incompleteness, structural differences, and false positives arising from variations in compilation environments. In this paper, we propose a novel Unsupervised Semantic Learning Model named SENSE for cross-platform vulnerability search. The model comprises two main components: semantic learner and graph learner. The semantic learner is pre-trained with a mask language task on a well-normalized binary corpus, enabling it to capture contextual semantic relations and generate block embedding that effectively encode the semantic features. In the graph learner, a gated graph neural network with a self-gating layer is adopted to eliminate redundant features and an adversarial loss is incorporated to enhance the robustness of function embedding across different compiler environments. Finally, SENSE is trained in an unsupervised manner using a batch-wise sampling strategy along with maximum mutual information loss. This encourages semantically similar functions to exhibit tighter embedding representations, thereby reducing false positives and improving search efficiency. Through extensive experiments, we have demonstrated that SENSE outperforms state-of-the-art methods in terms of binary search accuracy. Our results also reveal that SENSE is capable of generating robust function embedding that mitigate the differences arising from diverse architecture and optimization options.

DeepCPR: Deep Path Reasoning Using Sequence of User-Preferred Attributes for Conversational Recommendation

Conversational recommender systems (CRS) have garnered significant attention in academia and industry because of their ability to capture user preferences via system questions and user responses. Typically, in a CRS, reinforcement learning (RL) is utilized to determine the optimal timing for requesting attribute information or suggesting items. However, existing methods consider user-preferred attributes independently and ignore that attributes may be of different importance to the same user, in the attribute and item selection phases, which limits the accuracy and interpretability of CRS. Inspired by this, we propose deep conversational path reasoning (DeepCPR), which involves constructing a reasoning path on a graph with a series of user-favored attributes. It utilizes the attention mechanism to thoroughly examine the connections between these attributes and provide improved explanations for which attributes to inquire about or which items to recommend. In DeepCPR, two deep-learning-based modules are proposed to realize attribute and item selection. In the first module, the sequence of attributes confirmed by the user in conversation is encoded with a gated graph neural network to obtain the user’s long-term preference using a self-attention mechanism for the selection of candidate attributes. In the second module, a self-attention approach with more appropriate strategies is developed to dynamically select candidate items. In addition, to achieve fine-grained user preference modeling, a recurrent neural network is employed to aggregate the sequence of attributes that interact with the users. Numerous experimental evaluations conducted on four real CRS datasets show that the proposed method significantly outperforms existing advanced methods in terms of conversational recommendations.

LA-MGFM: A legal judgment prediction method via sememe-enhanced graph neural networks and multi-graph fusion mechanism

Legal Judgment Prediction (LJP) is a significant task of legal intelligence. Its objective is to predict the relevant law articles, charges, and terms of penalty based on fact descriptions of a criminal case. Existing methods have a drawback: they cannot effectively deal with charges confusion when using various granularity of law articles and predicting outcomes with limited data. In response to this challenge, we propose a solution: a graph neural network-based LJP method that utilizes a multi-graph fusion mechanism to fully and accurately integrate law article information. In detail, we begin by constructing five types of graphs for each case. In the phase of intra-graph information passing, we adopt a Sememe-enhanced Gated Graph Neural Networks to aggregate and update the node features by combining law articles and sememe information. For inter-graph information passing, we introduce a multi-graph fusion mechanism that merges the node features of the five graphs. Finally, we devise a graph readout function, which employs a classifier to derive the results of LJP. The results of our experiment on real-world datasets demonstrate that our method outperforms the current state-of-the-art approaches in our experimental metric.