Gated Graph Neural Network Research Articles

Fault localization method for power distribution systems based on gated graph neural networks

Fault localization is a key task on power systems operation and maintenance. When it comes to distribution networks, the problem is especially challenging due to the non-homogeneous characteristics and unique topology of each feeder. This paper presents a method based on gated graph neural network for automatic fault localization on distribution networks. The method aggregates problem data in a graph, where the feeder topology is represented by the graph links and nodes attributes can encapsulate any selected information such as operated devices, electrical characteristics and measurements at the point. The main advantage of the proposed solution is that it is immune to network reconfiguration and allows the use of a single trained model on multiple feeders. An experiment was conducted with faults simulated on 10 different feeders, all of them based on actual distribution feeders. The results shows that the model is able to generalize the correlations learned on training to correctly predict the fault region in most cases, even on a feeder it has not seen before.

Spatial–Temporal Relation Reasoning for Action Prediction in Videos

Action prediction in videos refers to inferring the action category label by an early observation of a video. Existing studies mainly focus on exploiting multiple visual cues to enhance the discriminative power of feature representation while neglecting important structure information in videos including interactions and correlations between different object entities. In this paper, we focus on reasoning about the spatial–temporal relations between persons and contextual objects to interpret the observed video part for predicting action categories. With this in mind, we propose a novel spatial–temporal relation reasoning approach that extracts the spatial relations between persons and objects in still frames and explores how these spatial relations change over time. Specifically, for spatial relation reasoning, we propose an improved gated graph neural network to perform spatial relation reasoning between the visual objects in video frames. For temporal relation reasoning, we propose a long short-term graph network to model both the short-term and long-term varying dynamics of the spatial relations with multi-scale receptive fields. By this means, our approach can accurately recognize the video content in terms of fine-grained object relations in both spatial and temporal domains to make prediction decisions. Moreover, in order to learn the latent correlations between spatial–temporal object relations and action categories in videos, a visual semantic relation loss is proposed to model the triple constraints between objects in semantic domain via VTransE. Extensive experiments on five public video datasets (i.e., 20BN-something-something, CAD120, UCF101, BIT-Interaction and HMDB51) demonstrate the effectiveness of the proposed spatial–temporal relation reasoning on action prediction.

Item trend learning for sequential recommendation system using gated graph neural network

Recommendation system, or recommender system, is widely used in online Web applications like e-commerce Web sites and movie review Web sites. Sequential recommender put more emphasis upon user’s short-term preference through exploiting information from its recent history. By incorporating the user short-term preference into the recommendation, the algorithm achieves a higher accuracy, which proves that a more accurate user portrait or representation boosts the performance to a great extent. Intuitionally, we seek to improve the current item representation modeling via incorporating the item trend information. Most of the recommendation models neglect the importance of the ever-changing item popularity. To this end, this paper introduces a novel sequential recommendation approach dubbed TRec. TRec learns the item trend information from the implicit user interaction history and incorporates the item trend information into the subsequent item recommendation tasks. After that, a self-attention mechanism is used for better representation. We also investigate alternative ways to model the proposed item trend representation; we evaluate two variant models which leverage the power of gated graph neural network upon the item trend representation modeling to boost the representation ability. We conduct extensive experiments with seven baseline methods on four benchmark datasets. The empirical results show that our proposed approach outperforms the state-of-the-art models as high as 18.2%. The experiment result displays the effectiveness in item trend information learning while with low computational complexity as well. Our study demonstrates the importance of item trend information in recommendation system.

Interactive prostate MR image segmentation based on ConvLSTMs and GGNN

Accurate segmentation of the prostate on magnetic resonance (MR) images plays an important role for prostate cancer diagnosis and treatment. Although many automated prostate segmentation methods have been proposed, the performance still faces several challenges, which includes large variability in prostate shape, unclear boundary, and complex intensity distribution. Therefore, the results obtained from the automated methods should be further refined by users to get a more accurate and reliable segmentation. In this paper, we propose an end-to-end interactive segmentation method to refine the automated results. A convolutional long short term memory (convLSTM) module and a gated graph neural network (GGNN) are presented in the proposed method for prostate segmentation in both automated and interactive manners. A boundary loss is proposed to train our model. We evaluated the proposed method on two public available datasets and one in–house dataset. Experimental results show that the proposed convLSTM module could obtain a DSC of 91.78% on the test dataset, which outperforms eight state-of-the-art methods. A further 1.5% improvements can be obtained by user interactions based on the GGNN. The segmentation time including user interactions and inference time was 2.3 min on average for segmenting one volume.

Session Recommendation Model Based on Context-Aware and Gated Graph Neural Networks.

The graph neural network (GNN) based approach has been successfully applied to session-based recommendation tasks. However, in the face of complex and changing real-world situations, the existing session recommendation algorithms do not fully consider the context information in user decision-making; furthermore, the importance of context information for the behavior model has been widely recognized. Based on this, this paper presents a session recommendation model based on context-aware and gated graph neural networks (CA-GGNNs). First, this paper presents the session sequence as data of graph structure. Second, the embedding vector representation of each item in the session graph is obtained by using the gated graph neural network (GGNN). In this paper, the GRU in GGNN is expanded to replace the input matrix and the state matrix in the conventional GRU with input context captured in the session (e.g., time, location, and holiday) and interval context (representing the proportion of the total session time of each item in the session). Finally, a soft attention mechanism is used to capture users' interests and preferences, and a recommendation list is given. The CA-GGNN model combines session sequence information with context information at each time. The results on the open Yoochoose and Diginetica datasets show that the model has significantly improved compared with the latest session recommendation methods.

Enhanced Natural Language Interface for Web-Based Information Retrieval

Database application is at the core of most web application systems such as web-based email, source codes repository management, public scientific data repository management, news portals, and publication repository of various fields. However, the usage of these database systems for data and information retrieval is severely limited because of lacking support for processing search queries expressed in a natural language (NL). Most web interfaces for databases today only take search queries entered in some form of logical combination of keywords or text strings, which restrict the scope and depth of what a web user really wants to search for, even though natural language based data or information retrieval has made significant advances in recent years. To overcome or at least to alleviate such limitation in web information services, we propose in this article an improved neural model based on an existing framework IRNet for NL query of databases, in which a representation of Gated Graph Neural Network (GGNN) is introduced to encode the database entities and relations. We also represent and use the database values in the prediction model to identify and match table and column names for automatic synthesize a correct SQL statement from a query expressed in a NL sentence. Experiments with a public dataset demonstrates the promising potential of our approach.

A knowledge-guided model of fitting detection in aerial transmission line images

Transmission lines are one of the most important infrastructures of energy Internet in China, and the effective detection of fittings is a necessary prerequisite to ensure their safety and stability. The fitting detection method based on classic deep learning only focuses on a single region and extracts the visual features of the region proposal into the classifier to identify the target, but never considers the connection between fittings. There is a certain regularity between fittings when constructing a complete and intact transmission line, interdependent and interact with each other. Therefore, we propose a knowledge-guided aerial transmission line image fitting detection model (KGFD) which uses two modules to learn the regularity of the fittings. The first module is an implicit module, starting from the spatial layout of the fittings on the image and taking the relative geometric features of the region proposal as the input knowledge of the spatial position of the fittings on the image. The second module is an explicit one. By introducing the prior knowledge of the fittings expressed in co-occurrence mode and using the region proposal as the node while the prior knowledge as the edge, a knowledge graph is constructed. The information spreads on the knowledge graph via a gated graph neural network, which realizes a combination of the prior knowledge of the fitting with the features of the region proposal. These two modules are added to the deep learning framework to detect the fitting targets, and the typical fitting data set of aerial transmission lines are used for testing. The AP value increased by 4.4% when IoU={0.5:0.95} and the AP value by 3.9% when IoU=0.5 compared with those of Faster R-CNN classification.

Where to go? Predicting next location in IoT environment

Next location prediction has aroused great interests in the era of internet of things (IoT). With the ubiquitous deployment of sensor devices, e.g., GPS and Wi-Fi, IoT environment offers new opportunities for proactively analyzing human mobility patterns and predicting user’s future visit in low cost, no matter outdoor and indoor. In this paper, we consider the problem of next location prediction in IoT environment via a session-based manner. We suggest that user’s future intention in each session can be better inferred for more accurate prediction if patterns hidden inside both trajectory and signal strength sequences collected from IoT devices can be jointly modeled, which however existing state-of-the-art methods have rarely addressed. To this end, we propose a trajectory and sIgnal sequence (TSIS) model, where the trajectory transition regularities and signal temporal dynamics are jointly embedded in a neural network based model. Specifically, we employ gated recurrent unit (GRU) for capturing the temporal dynamics in the multivariate signal strength sequence. Moreover, we adapt gated graph neural networks (gated GNNs) on location transition graphs to explicitly model the transition patterns of trajectories. Finally, both the low-dimensional representations learned from trajectory and signal sequence are jointly optimized to construct a session embedding, which is further employed to predict the next location. Extensive experiments on two real-world Wi-Fi based mobility datasets demonstrate that TSIS is effective and robust for next location prediction compared with other competitive baselines.

Handling Missing Sensors in Topology-Aware IoT Applications with Gated Graph Neural Network

Reliable data collection, transmission, and delivery on Internet of Things (IoT) systems is crucial in order to provide high-quality intelligent services. However, sensor data delivery can be interrupted for various reasons, such as sensor malfunction, network failures, and external attacks. Thus, only data from a partial set of sensors may be available. We call it the missing sensor problem. This problem can lead to severe performance degradation at inference time by neural-network-based recognition models trained on the complete sensor set. This paper enhances the robustness of neural network models to the missing sensor problem by introducing a novel feature reconstruction module, named the graph recovery module, that handles missing sensors directly inside the network. Specifically, we consider topology-aware IoT applications, where sensors are placed on a physically interconnected network. We design a novel neural message passing mechanism that logically mimics physical network topology, based on recent advances in graph neural networks (GNNs). We rely on a spatial locality assumption, where only correlations between physically connected sensors are explicitly explored. When encountering missing sensors, information is passed from available sensors to missing sensors to be used to reconstruct their features. Moreover, at each message passing step, we utilize a gating mechanism inspired by Gated Recurrent Units (GRUs) to automatically control information flow between available sensors and missing sensors. We empirically evaluate the reconstruction performance of the graph recovery module with two representative IoT applications; human activity recognition (HAR) and electroencephalogram (EEG)-based motor-imagery classification, on three public datasets. Two different backbone networks are utilized for the tasks. Our design is shown to effectively maintain model performance, suffering only 7% to 18% accuracy loss when as much as 90% of sensors are removed, compared to a drop of 15% to 47% in the accuracy of competing state-of-the-art algorithms under the same conditions. The accuracy gap is largest when more sensors are missing.

Explainable predictive business process monitoring using gated graph neural networks

ABSTRACT Predictive business process monitoring (PBPM) is a class of techniques designed to forecast future behaviour of a running process instance or the value of process-related metrics like times and frequencies. PBPM systems support process workers and process managers in making operational decisions. State-of-the-art PBPM systems apply deep-learning techniques with multiple hidden layers to infer from data which makes it difficult for system users to understand why a prediction was made. However, the user needs to see deeper causes to identify intervention mechanisms that secure process performance. The main contribution of this paper is a technique that makes a prediction more explainable by visualising how much the different activities included in a process impacted the prediction. This work is the first to use gated graph neural networks (GGNNs) to make decisions more explainable, and it’s also GGNN’s first application to PBPM. We use a process event data set to demonstrate our approach.

Rumor detection based on propagation graph neural network with attention mechanism

Rumors on social media have always been an important issue that seriously endangers social security. Researches on timely and effective detection of rumors have aroused lots of interest in both academia and industry. At present, most existing methods identify rumors based solely on the linguistic information without considering the temporal dynamics and propagation patterns. In this work, we aim to solve rumor detection task under the framework of representation learning. We first propose a novel way to construct the propagation graph by following the propagation structure (who replies to whom) of posts on Twitter. Then we propose a gated graph neural network based algorithm called PGNN, which can generate powerful representations for each node in the propagation graph. The proposed PGNN algorithm repeatedly updates node representations by exchanging information between the neighbor nodes via relation paths within a limited time steps. On this basis, we propose two models, namely GLO-PGNN (rumor detection model based on the global embedding with propagation graph neural network) and ENS-PGNN (rumor detection model based on the ensemble learning with propagation graph neural network). They respectively adopt different classification strategies for rumor detection task, and further improve the performance by including attention mechanism to dynamically adjust the weight of each node in the propagation graph. Experiments on a real-world Twitter dataset demonstrate that our proposed models achieve much better performance than state-of-the-art methods both on the rumor detection task and early detection task.

Attentive Gated Graph Neural Network for Image Scene Graph Generation

Image scene graph is a semantic structural representation which can not only show what objects are in the image, but also infer the relationships and interactions among them. Despite the recent success in object detection using deep neural networks, automatically recognizing social relations of objects in images remains a challenging task due to the significant gap between the domains of visual content and social relation. In this work, we translate the scene graph into an Attentive Gated Graph Neural Network which can propagate a message by visual relationship embedding. More specifically, nodes in gated neural networks can represent objects in the image, and edges can be regarded as relationships among objects. In this network, an attention mechanism is applied to measure the strength of the relationship between objects. It can increase the accuracy of object classification and reduce the complexity of relationship classification. Extensive experiments on the widely adopted Visual Genome Dataset show the effectiveness of the proposed method.

Semantic Relationships Guided Representation Learning for Facial Action Unit Recognition

Facial action unit (AU) recognition is a crucial task for facial expressions analysis and has attracted extensive attention in the field of artificial intelligence and computer vision. Existing works have either focused on designing or learning complex regional feature representations, or delved into various types of AU relationship modeling. Albeit with varying degrees of progress, it is still arduous for existing methods to handle complex situations. In this paper, we investigate how to integrate the semantic relationship propagation between AUs in a deep neural network framework to enhance the feature representation of facial regions, and propose an AU semantic relationship embedded representation learning (SRERL) framework. Specifically, by analyzing the symbiosis and mutual exclusion of AUs in various facial expressions, we organize the facial AUs in the form of structured knowledge-graph and integrate a Gated Graph Neural Network (GGNN) in a multi-scale CNN framework to propagate node information through the graph for generating enhanced AU representation. As the learned feature involves both the appearance characteristics and the AU relationship reasoning, the proposed model is more robust and can cope with more challenging cases, e.g., illumination change and partial occlusion. Extensive experiments on the two public benchmarks demonstrate that our method outperforms the previous work and achieves state of the art performance.