Gated Graph Neural Network Research Articles

LA-MGFM: A legal judgment prediction method via sememe-enhanced graph neural networks and multi-graph fusion mechanism

Legal Judgment Prediction (LJP) is a significant task of legal intelligence. Its objective is to predict the relevant law articles, charges, and terms of penalty based on fact descriptions of a criminal case. Existing methods have a drawback: they cannot effectively deal with charges confusion when using various granularity of law articles and predicting outcomes with limited data. In response to this challenge, we propose a solution: a graph neural network-based LJP method that utilizes a multi-graph fusion mechanism to fully and accurately integrate law article information. In detail, we begin by constructing five types of graphs for each case. In the phase of intra-graph information passing, we adopt a Sememe-enhanced Gated Graph Neural Networks to aggregate and update the node features by combining law articles and sememe information. For inter-graph information passing, we introduce a multi-graph fusion mechanism that merges the node features of the five graphs. Finally, we devise a graph readout function, which employs a classifier to derive the results of LJP. The results of our experiment on real-world datasets demonstrate that our method outperforms the current state-of-the-art approaches in our experimental metric.

Semantic Clone Detection Based on Code Feature Fusion Learning

Code clones are duplicated code snippets that significantly threaten software maintenance and the public corpora of code representation learning. Traditionally, code context and its structure information abstract syntax tree (AST), control flow graph (CFG) are typical representations of source code, and context-based models and structure-based models contributed significantly to the development of code clone detection. In this paper, we present a hybrid embedding model for code clone detection (HEM-CCD), a fusion method of token sequential information and graph-based structure information. We insert tokens’ global context information encoded by a bi-directional recurrent neural network into the AST-based graph for comprehensive code semantic representation. Then, feeding the graph into a gated graph neural network we generate code semantic vectors for similarity evaluation. We have implemented our model on two public clone datasets (BigCloneBench and GoogleCodeJam), and the results indicate that HEM-CCD outperforms several state-of-the-art approaches.

VulGraB: Graph‐embedding‐based code vulnerability detection with bi‐directional gated graph neural network

AbstractCode vulnerabilities can have serious consequences such as system attacks and data leakage, making it crucial to perform code vulnerability detection during the software development phase. Deep learning is an emerging approach for vulnerability detection tasks. Existing deep learning‐based code vulnerability detection methods are usually based on word2vec embedding of linear sequences of source code, followed by code vulnerability detection through RNNs network. However, such methods can only capture the superficial structural or syntactic information of the source code text, which is not suitable for modeling the complex control flow and data flow and miss edge information in the graph structure constructed by the source code, with limited effect of neural network model. To solve the above problems, this article proposes a code vulnerability detection method, named VulGraB, which is based on graph embedding and bidirectional gated graph neural networks. VulGraB uses node2vec to convert the program‐dependent graphs into graph embeddings of the code, which contain rich structure information of the source code, improving the ability of features to express nonlinear information to a certain extent. Then the BiGGNN is used for training, and finally the accuracy of the detection results is evaluated using target program. The bi‐directional gated neural network utilizes a bi‐directional recurrent structure, which is beneficial to global information aggregation. The experimental results show that the accuracy of VulGraB is significantly improved over the baseline models on two datasets, with F1 scores of 85.89% and 97.24% being the highest, demonstrating that VulGraB consistently outperforms other effective vulnerability detection models.

VDoTR: Vulnerability detection based on tensor representation of comprehensive code graphs

Code vulnerability detection has long been a critical issue due to its potential threat to computer systems. It is imperative to detect source code vulnerabilities in software and remediate them to avoid cyber attacks. To automate detection and reduce labor costs, many deep learning-based methods have been proposed. However, these approaches have been found to be either ineffective in detecting multiple classes of vulnerabilities or limited by treating original source code as a natural language sequence without exploiting the structural information of code. In this paper, we propose VDoTR, a model that leverages a new tensor representation of comprehensive code graphs, including AST, CFG, DFG, and NCS, to detect multiple types of vulnerabilities. Firstly, a tensor structure is introduced to represent the structured information of code, which deeply captures code features. Secondly, a new Circle Gated Graph Neural Network (CircleGGNN) is designed based on tensor for hidden state embedding of nodes. CircleGGNN can perform heterogeneous graph information fusion more directly and effectively. Lastly, a 1-D convolution-based output layer is applied to hidden embedding features for classification. The experimental results demonstrate that the detection performance of VDoTR is superior to other approaches with higher accuracy, precision, recall, and F1-measure on multiple datasets for vulnerability detection. Moreover, we illustrate which code graph contributes the most to the performance of VDoTR and which code graph is more sensitive to represent vulnerability features for different types of vulnerabilities through ablation experiments.

DeGraphCS : Embedding Variable-based Flow Graph for Neural Code Search

With the rapid increase of public code repositories, developers maintain a great desire to retrieve precise code snippets by using natural language. Despite existing deep learning-based approaches that provide end-to-end solutions (i.e., accept natural language as queries and show related code fragments), the performance of code search in the large-scale repositories is still low in accuracy because of the code representation (e.g., AST) and modeling (e.g., directly fusing features in the attention stage). In this paper, we propose a novel learnable de ep G raph for C ode S earch (called deGraphCS ) to transfer source code into variable-based flow graphs based on an intermediate representation technique, which can model code semantics more precisely than directly processing the code as text or using the syntax tree representation. Furthermore, we propose a graph optimization mechanism to refine the code representation and apply an improved gated graph neural network to model variable-based flow graphs. To evaluate the effectiveness of deGraphCS , we collect a large-scale dataset from GitHub containing 41,152 code snippets written in the C language and reproduce several typical deep code search methods for comparison. The experimental results show that deGraphCS can achieve state-of-the-art performance and accurately retrieve code snippets satisfying the needs of the users.

Learning-Based Topology Optimization of Power Networks

Transitions of power networks demand more powerful solution approaches for optimal transmission switching (OTS) and distribution network reconfiguration (DNR) to tackle complicated cases with high computational efficiency. In this paper, we propose a learning-based solution approach for OTS and DNR, which is established based on an identified common pattern behind existing heuristic algorithms. The original hand-crafted and short-sighted evaluation function is substituted by a parameterized function whose architecture is well-designed by mainly exploiting the gated graph neural network and multilayer perceptrons. For learning the parameterized function that lacks training labels, we further design a learning algorithm that combines the double deep Q-network algorithm and multi-step learning. Then, with the learned parameterized function, the learning-based heuristic algorithm is developed by embedding a feasibility recovery procedure. Finally, numerical studies demonstrate the superiority of the learning-based heuristic algorithm in both solution optimality and computational efficiency.

Molecule generation toward target protein (SARS-CoV-2) using reinforcement learning-based graph neural network via knowledge graph.

AI-driven approaches are widely used in drug discovery, where candidate molecules are generated and tested on a target protein for binding affinity prediction. However, generating new compounds with desirable molecular properties such as Quantitative Estimate of Drug-likeness (QED) and Dopamine Receptor D2 activity (DRD2) while adhering to distinct chemical laws is challenging. To address these challenges, we proposed a graph-based deep learning framework to generate potential therapeutic drugs targeting the SARS-CoV-2 protein. Our proposed framework consists of two modules: a novel reinforcement learning (RL)-based graph generative module with knowledge graph (KG) and a graph early fusion approach (GEFA) for binding affinity prediction. The first module uses a gated graph neural network (GGNN) model under the RL environment for generating novel molecular compounds with desired properties and a custom-made KG for molecule screening. The second module uses GEFA to predict binding affinity scores between the generated compounds and target proteins. Experiments show how fine-tuning the GGNN model under the RL environment enhances the molecules with desired properties to generate valid and unique compounds using different scoring functions. Additionally, KG-based screening reduces the search space of generated candidate molecules by while retaining of promising binding molecules against SARS-CoV-2 protein, i.e., 3C-like protease (3CLpro). We achieved a binding affinity score of 8.185 from the top rank of generated compound. In addition, we compared top-ranked generated compounds to Indinavir on different parameters, including drug-likeness and medicinal chemistry, for qualitative analysis from a drug development perspective. The online version contains supplementary material available at 10.1007/s13721-023-00409-2.

A Gated Graph Neural Network With Attention for Text Classification Based on Coupled P Systems

A Gated Graph Neural Network With Attention for Text Classification Based on Coupled P Systems

Disentangled Graph Neural Networks for Session-Based Recommendation

Session-based recommendation (SBR) has drawn increasingly research attention in recent years, due to its great practical value by only exploiting the limited user behavior history in the current session. The key of SBR is to accurately infer the anonymous user purpose in a session which is typically represented as session embedding, and then match it with the item embeddings for the next item prediction. Existing methods typically learn the session embedding at the item level, namely, aggregating the embeddings of items with or without assigned attention weights to items. However, they ignore the fact that a user's intent on adopting an item is driven by certain factors of the item (e.g., the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">leading actors</i> of an movie). In other words, they have not explored finer-granularity interests of users at the factor level to generate the session embedding, leading to sub-optimal performance. To address the problem, we propose a novel method called Disentangled Graph Neural Network (Disen-GNN) to capture the session purpose with the consideration of factor-level attention on each item. Specifically, we first employ the disentangled learning technique to cast item embeddings into the embeddings of multiple factors, and then use the gated graph neural network (GGNN) to learn the embedding factor-wisely based on the item adjacent similarity matrix computed for each factor. Moreover, the distance correlation is adopted to enhance the independence between each pair of factors. After representing each item with independent factors, an attention mechanism is designed to learn user intent to different factors of each item in the session. The session embedding is then generated by aggregating the item embeddings with attention weights of each item's factors. To this end, our model takes user intents at the factor level into account to infer the user purpose in a session. Extensive experiments on three benchmark datasets demonstrate the superiority of our method over existing methods.

"Reconfiguration of Automated Manufacturing Systems Using Gated Graph Neural Networks"

"Reconfiguration of Automated Manufacturing Systems Using Gated Graph Neural Networks"

"Reconfiguration of Automated Manufacturing Systems Using Gated Graph Neural Networks"

"Reconfiguration of Automated Manufacturing Systems Using Gated Graph Neural Networks"

Reinforcement learning-driven deep question generation with rich semantics

Deep question generation (DQG) refers to generating a complex question from different sentences in context. Existing methods mainly focus on enhancing information extraction based on the encoder–decoder neural networks though they cannot perform well in DQG tasks. To address this issue, we consider combining reinforcement learning with semantic-rich information to generate deep questions in this paper. In particular, we propose a Semantic-Rich Reinforcement Learning Deep Question Generation (SRL-DQG) model, which better utilizes the semantic graphs of document representations based on the Gated Graph Neural Network (GGNN). In order to generate high-quality questions, we also optimize specific objectives via reinforcement learning with consideration of four evaluation factors including naturality, relevance, answerability, and difficulty. Empirical evaluations demonstrate that our SRL-DQG effectively improves the quality of generated questions and achieves superior performance than existing methods in terms of multiple performance metrics. Specifically, we show that several BLEU-n scores were improved by 3.5% to 10% after running SRL-DQG on 6072 samples of HotPotQA.

Handling missing data in well-log curves with a gated graph neural network

Well logging is a common method that is used to obtain the rock properties of a formation. It is relatively frequent, however, that log information is incomplete due to cost limitations or borehole problems. Existing models predict missing well logs from a fixed combination of other available well logs. However, the missing well logs vary from well to well. We have proposed using a gated graph neural network (GNN) to handle the missing values in well-log curves. It takes sequential data, predicting each missing measurement in the data not only using other available variables measured at the same depth but also available measurements of neighboring observations. Meanwhile, the missing well logs and available well logs could be any possible combinations as long as they are mutually exclusive. This approach has two advantages: (1) the gated GNN does not need to build a specific model for each missing measurement or from every possible combination of available measurements and (2) it can be integrated into the training process of the following predictive model to perform classification tasks. We evaluate the gated GNN model along with two other models: the GRAPE model and the multiple imputation by chained equations (MICE)-gated recurrent unit (GRU) model, on a data set from the North Sea to perform a missing feature imputation task and a lithofacies identification task. The GRAPE model also is a graph-based model, and it predicts values for each missing measurement from available variables measured at the same depth. The MICE-GRU model is a combination of the MICE algorithm and GRU, which handles the feature imputation procedure and the lithofacies identification procedure separately. Our experiments find that the gated GNN model outperforms the MICE algorithm and the GRAPE model on the missing feature imputation task. For the lithofacies identification task, the gated GNN model also provides comparable results to the MICE-GRU model, and they both outperform the GRAPE model.

Reconfiguration of Automated Manufacturing Systems Using Gated Graph Neural Networks

To deal with the unpredictability of dynamic markets, automated manufacturing systems rely ontheir capacity to adapt and change. With the need for more personalized and high-quality goods,the complexity of these systems evolves, prompting more agile and adaptable techniques. Toenable dynamic as well as on systems reconfiguration aimed at responding swiftly to productchanges by providing more efficient services. To increase production in response to marketdemand and meet the referred requirements, this proposed study employs Machine LearningTechniques for the Reconfiguration of Automated Manufacturing Systems. Gated Graph NeuralNetwork (GGNN) based prediction model is generated using graph instances as input, and theprediction model provides a result for each graph instance, as well as activity level relevanceand ratings for the relevant needs such as model accuracy and validation. For better use of themodel effectiveness by the proposed methodology for the final model is validated for cost, time,and productivity.

A graph-based pivotal semantic mining framework for rumor detection

As the gathering place for modern content sharing, social platforms not only provide places for people to obtain information and express opinions, but also provide hidden channels for the generation and dissemination of rumors. Rumor detection is an important and challenging task for mining fake information in social networks. Previous methods utilized sequential models to embed semantic features, which are not comprehensive for mining pivotal semantic information and easy to ignore discontinuous dependencies. However, excessive mining of information content will lead to the acquisition of some redundant information, whose existence will affect the extraction of useful features and the detection ability of the model. To address these issues, this paper offers a graph-based pivotal semantic mining framework. Specifically, we model the content information as a graph structure, learn the semantic dependencies across segments through a gated graph neural network, and co-learn by combining the propagation features of rumors. Furthermore, in order to highlight the precious essential semantic information, a shared unit cell is offered to reduce the influence of redundant information. Experimental results on realworld datasets show that the proposed method exceeds existing methods in terms of benchmark testing.

Causality-based CTR prediction using graph neural networks

As a prevalent problem in online advertising, CTR prediction has attracted plentiful attention from both academia and industry. Recent studies have been reported to establish CTR prediction models in the graph neural networks (GNNs) framework. However, most of GNNs-based models handle feature interactions in a complete graph, while ignoring causal relationships among features, which results in a huge drop in the performance on out-of-distribution data. This paper is dedicated to developing a causality-based CTR prediction model in the GNNs framework (Causal-GNN) integrating representations of feature graph, user graph and ad graph in the context of online advertising. In our model, a structured representation learning method (GraphFwFM) is designed to capture high-order representations on feature graph based on causal discovery among field features in gated graph neural networks (GGNNs), and GraphSAGE is employed to obtain graph representations of users and ads. Experiments conducted on three public datasets demonstrate the superiority of Causal-GNN in AUC and Logloss and the effectiveness of GraphFwFM in capturing high-order representations on causal feature graph.

Session Recommendation Based on Edge Information Clustering

Traditional session recommendation mainly uses the time sequence of users clicking items to construct a user session graph, which often ignores the similarity and differences between user groups. To improve the effect of recommendation, an E-SGNN (E-SGNN, Edge-Session Graph Neural Network) method combining edge information clustering and session recommendation is proposed. Firstly, similar users are clustered by edge information and divided into different session user groups. After extracting the data features of the user site relationship graph in the session, it is reset and updated through the gated graph neural network (GGNN); Secondly, a self-attention mechanism is introduced to adjust the proportion of users’ current preference and historical preference; Finally, the ranking score is obtained through linear transformation and softmax classifier. The higher the score, the more obvious the user’s preference for the item. Experiments show that compared with session-based graph neural network and cross-session information recommendation, the E-SGNN algorithm proposed in this paper has a significant improvement in recall rate and average reciprocal ranking. When the three edge parameters are combined, the recall rate reaches 98.97% and the average reciprocal ranking reaches 45.77%.

Graph-based detection for false data injection attacks in power grid

False data injection attack (FDIA) is the main network attack type threatening power system. FDIA affect the accuracy of data by modifying the measured values of measuring equipment. When the power grid topology has not changed, the data-driven detection method has high detection accuracy. However, the power grid topology changes when faults, maintenance or adjustment in power flow distribution occur in the power system. The data-driven detection method can not capture the spatial feature changes, and the detection accuracy decreases. Considering the changes of power grid topology, a false data detection method based on graph neural network is proposed in this paper, which extracts the spatial features of power grid topology information and operation data through gated graph neural network (GGNN). To enhance node representation in this method, the attention mechanism is adopted to assign aggregation weights to neighbor nodes. This method is tested on IEEE 14-bus and IEEE 118-bus systems. The experimental simulation results show that compared with other data-driven detection methods, the proposed method can significantly improve the detection accuracy under the changes of power grid topology.

Combine sliced joint graph with graph neural networks for smart contract vulnerability detection

Smart contract security has drawn extensive attention in recent years because of the enormous economic losses caused by vulnerabilities. Even worse, fixing bugs in a deployed smart contract is difficult, so developers must detect security vulnerabilities in a smart contract before deployment. Existing smart contract vulnerability detection efforts heavily rely on fixed rules defined by experts, which are inefficient and inflexible.To overcome the limitations of existing vulnerability detection approaches, we propose a GNN based approach for smart contract vulnerability detection. First, we construct a graph representation for a smart contract function with syntactic and semantic features by combining abstract syntax tree (AST), control flow graph (CFG), and program dependency graph (PDG). To further strengthen the presentation ability of our approach, we perform program slicing to normalize the graph and eliminate the redundant information unrelated to vulnerabilities. Then, we use a Bidirectional Gated Graph Neural-Network model with hybrid attention pooling to identify potential vulnerabilities in smart contract functions. Empirical results show that our approach can achieve 89.2% precision and 92.9% recall in smart contract vulnerability detection on our dataset and reveal the effectiveness and efficiency of our approach.

Traffic-GGNN: Predicting Traffic Flow via Attentional Spatial-Temporal Gated Graph Neural Networks

Recent spatial-temporal graph-based deep learning methods for Traffic Flow Prediction (TFP) problems have shown superior performance in modeling higher-level spatial interactions and temporal correlations. However, most of these methods suffer from post-fusion efficiency difficulty caused by separate explorations of the spatial communications and the temporal dependencies, which could result in delayed and biased predictions. To address that, we propose a Traffic Gated Graph Neural Networks (Traffic-GGNN) for real-time-fused spatial-temporal representation modeling. Firstly, we adopt bidirectional message passing to capture the location-wise spatial interactions. Secondly, we apply a GRU-based module to explore and aggregate the spatial interactions with the temporal correlations in a real-time fusion way. Lastly, we introduce a self-attention mechanism to reweight the location-based importance and produce the final prediction. Moreover, our proposed model allows end-to-end training thus it is easy to scale to diverse types of traffic datasets and yield better efficiency and effectiveness on three real-world datasets (SZ-taxi, Los-loop, and PEMS-BAY).