Graph Convolutional Network Research Articles

Spatio-temporal deep learning framework for pedestrian intention prediction in urban traffic scenes

Pedestrian intent prediction is an essential task for ensuring the safety of pedestrians and vehicles on the road. This task involves predicting whether a pedestrian intends to cross a road or not based on their behavior and surrounding environment. Previous studies have explored feature-based machine learning and vision-based deep learning models for this task but these methods have limitations in capturing the global spatio-temporal context and fusing different features of data effectively. To address these issues, we propose a novel hybrid framework HSTGCN for pedestrian intent prediction that combines spatio-temporal graph convolutional neural networks (STGCN) and long short-term memory (LSTM) networks. The proposed framework utilizes the strengths of both models by fusing multiple features, including skeleton pose, trajectory, height, orientation, and ego-vehicle speed, to predict their intentions accurately. The framework’s performance have been evaluated on the JAAD benchmark dataset and the results show that it outperforms the state-of-the-art methods. The proposed framework has potential applications in developing intelligent transportation systems, autonomous vehicles, and pedestrian safety technologies. The utilization of multiple features can significantly improve the performance of the pedestrian intent prediction task.

Government chatbot: Empowering smart conversations with enhanced contextual understanding and reasoning

Currently, an increasing number of governments have adopted question answering systems (QASs) in public service delivery. As some citizens with limited information literacy often express their questions vaguely when interacting with a chatbot, it is necessary to improve the contextual understanding and reasoning ability of government chatbots (G-chatbots). This goal can be achieved through the optimisation of the matching between question, answer and context. By incorporating the Relational Graph Convolutional Networks (R-GCNs) and fuzzy logic, this study proposes a multi-turn dialogue model that introduces a re-question mechanism and a subgraph matching algorithm. The experiment results show that the model can improve the contextual reasoning ability of G-chatbots by about 10% and generate answers in a more explainable way. This study innovatively integrates a question–answer–context matching approach, re-question mechanism into the MTRF-G-chatbot model, reducing barriers to citizens’ access to government services and enhancing contextual reasoning abilities.

Research on decoupled adaptive graph convolution networks based on skeleton data for action recognition

Research on decoupled adaptive graph convolution networks based on skeleton data for action recognition

Emergency voltage control strategy for power system transient stability enhancement based on edge graph convolutional network reinforcement learning

Emergency control is essential for maintaining the stability of power systems, serving as a key defense mechanism against the destabilization and cascading failures triggered by faults. Under-voltage load shedding is a popular and effective approach for emergency control. However, with the increasing complexity and scale of power systems and the rise in uncertainty factors, traditional approaches struggle with computation speed, accuracy, and scalability issues. Deep reinforcement learning holds significant potential for the power system decision-making problems. However, existing deep reinforcement learning algorithms have limitations in effectively leveraging diverse operational features, which affects the reliability and efficiency of emergency control strategies. This paper presents an innovative approach for real-time emergency voltage control strategies for transient stability enhancement through the integration of edge-graph convolutional networks with reinforcement learning. This method transforms the traditional emergency control optimization problem into a sequential decision-making process. By utilizing the edge-graph convolutional neural network, it efficiently extracts critical information on the correlation between the power system operation status and node branch information, as well as the uncertainty factors involved. Moreover, the clipped double Q-learning, delayed policy update, and target policy smoothing are introduced to effectively solve the issues of overestimation and abnormal sensitivity to hyperparameters in the deep deterministic policy gradient algorithm. The effectiveness of the proposed method in emergency control decision-making is verified by the IEEE 39-bus system and the IEEE 118-bus system.

Molecular Classification with Graph ConvolutionalNetworks: Exploring the MUTAG Dataset for Mutagenicity Prediction

This paper presents the implementation of a Graph Convolutional Network (GCN) for the classification of chemical compounds using the MUTAG dataset, which consists of 188 ni- troaromatic compounds labeled according to their mutagenicity. The GCN model leverages the inherent graph structure of molec-ular data to capture and learn from the relationships between atoms and bonds, represented as nodes and edges, respectively. By utilizing three graph convolutional layers followed by a global mean pooling layer, the model effectively aggregates node features to generate meaningful graph-level representations. The model was trained using the Adam optimizer with a learning rate of 0.01, and cross-entropy loss was employed to supervise the classification task. The results demonstrate the efficacy of GCNs in graph classification tasks, with the model achieving a training accuracy of 79.33% and a test accuracy of 76.32%. This study highlights the potential of GCNs in cheminformatics and other domains where graph-structured data is prevalent, paving the way for further exploration and application of advanced graph neural networks in similar tasks.

Development of message passing-based graph convolutional networks for classifying cancer pathology reports

BackgroundApplying graph convolutional networks (GCN) to the classification of free-form natural language texts leveraged by graph-of-words features (TextGCN) was studied and confirmed to be an effective means of describing complex natural language texts. However, the text classification models based on the TextGCN possess weaknesses in terms of memory consumption and model dissemination and distribution. In this paper, we present a fast message passing network (FastMPN), implementing a GCN with message passing architecture that provides versatility and flexibility by allowing trainable node embedding and edge weights, helping the GCN model find the better solution. We applied the FastMPN model to the task of clinical information extraction from cancer pathology reports, extracting the following six properties: main site, subsite, laterality, histology, behavior, and grade.ResultsWe evaluated the clinical task performance of the FastMPN models in terms of micro- and macro-averaged F1 scores. A comparison was performed with the multi-task convolutional neural network (MT-CNN) model. Results show that the FastMPN model is equivalent to or better than the MT-CNN.ConclusionsOur implementation revealed that our FastMPN model, which is based on the PyTorch platform, can train a large corpus (667,290 training samples) with 202,373 unique words in less than 3 minutes per epoch using one NVIDIA V100 hardware accelerator. Our experiments demonstrated that using this implementation, the clinical task performance scores of information extraction related to tumors from cancer pathology reports were highly competitive.

ABNet: AI-Empowered Abnormal Action Recognition Method for Laboratory Mouse Behavior.

The automatic recognition and quantitative analysis of abnormal behavior in mice play a crucial role in behavioral observation experiments in neuroscience, pharmacology, and toxicology. Due to the challenging definition of abnormal behavior and difficulty in collecting training samples, directly applying behavior recognition methods to identify abnormal behavior is often infeasible. This paper proposes ABNet, an AI-empowered abnormal action recognition approach for mice. ABNet utilizes an enhanced Spatio-Temporal Graph Convolutional Network (ST-GCN) as an encoder; ST-GCN combines graph convolution and temporal convolution to efficiently capture and analyze spatio-temporal dynamic features in graph-structured data, making it suitable for complex tasks such as action recognition and traffic prediction. ABNet trains the encoding network with normal behavior samples, then employs unsupervised clustering to identify abnormal behavior in mice. Compared to the original ST-GCN network, the method significantly enhances the capabilities of feature extraction and encoding. We conduct comprehensive experiments on the Kinetics-Skeleton dataset and the mouse behavior dataset to evaluate and validate the performance of ABNet in behavior recognition and abnormal motion detection. In the behavior recognition experiments conducted on the Kinetics-Skeleton dataset, ABNet achieves an accuracy of 32.7% for the top one and 55.2% for the top five. Moreover, in the abnormal behavior analysis experiments conducted on the mouse behavior dataset, ABNet achieves an average accuracy of 83.1%.

Prompt-supervised dynamic attention graph convolutional network for skeleton-based action recognition

Skeleton-based action recognition is a core task in the field of video understanding. Skeleton sequences are characterized by high information density, low redundancy, and clear structural information, thereby facilitating the analysis of complex relationships among human behaviors more readily than other modalities. Although existing studies have encoded skeleton data and achieved positive outcomes, they have often overlooked the precise high-level semantic information inherent in the action descriptions. To address this issue, this paper proposes a prompt-supervised dynamic attention graph convolutional network (PDA-GCN). Specifically, the PDA-GCN incorporates a prompt supervision (PS) module that leverages a pre-trained large-scale language model (LLM) as a knowledge engine and retains the generated text features as prompts to provide additional supervision during model training, enhancing the model’s ability to discern analogous actions with negligible computational cost. In addition, for the purpose of bolstering the learning of discriminative features, a dynamic attention graph convolution (DA-GC) module is presented. This module utilizes self-attention mechanism to adaptively infer intrinsic relationships between joints and integrates dynamic convolution to strengthen the emphasis on local information. This dual focus on both global context and local details further amplifies the efficiency and effectiveness of the model. Extensive experiments, conducted on the widely-used skeleton-based action recognition datasets NTU RGB+D 60 and NTU RGB+D 120, demonstrate that the PDA-GCN surpasses known state-of-the-art methods, achieving accuracies of 93.4% on the NTU RGB+D 60 cross-subject split and 90.7% on the NTU RGB+D 120 cross-subject split.

A parallel neural networks for emotion recognition based on EEG signals

Our study proposes a novel Parallel Temporal–Spatial-Frequency Neural Network (PTSFNN) for emotion recognition. The network processes EEG signals in the time, frequency, and spatial domains simultaneously to extract discriminative features. Despite its relatively simple architecture, the proposed model achieves superior performance. Specifically, PTSFNN first applies wavelet transform to the raw EEG signals and then reconstructs the coefficients based on frequency hierarchy, thereby achieving frequency decomposition. Subsequently, the core part of the network performs three independent parallel convolution operations on the decomposed signals, including a novel graph convolutional network. Finally, an attention mechanism-based post-processing operation is designed to effectively enhance feature representation. The features obtained from the three modules are concatenated for classification, with the cross-entropy loss function being adopted. To evaluate the model’s performance, extensive experiments are conducted on the SEED and SEED-IV public datasets. The experimental results demonstrate that PTSFNN achieves excellent performance in emotion recognition tasks, with classification accuracies of 87.63% and 74.96%, respectively. Comparative experiments with previous state-of-the-art methods confirm the superiority of our proposed model, which can efficiently extract emotion information from EEG signals.

Context-Aware DGCN-Based Ship Formation Recognition in Remote Sensing Images

Ship detection and formation recognition in remote sensing have increasingly garnered attention. However, research remains challenging due to arbitrary orientation, dense arrangement, and the complex background of ships. To enhance the analysis of ship situations in channels, we model the ships as the key points and propose a context-aware DGCN-based ship formation recognition method. First, we develop a center point-based ship detection subnetwork, which employs depth-separable convolution to reduce parameter redundancy and combines coordinate attention with an oriented response network to generate direction-invariant feature maps. The center point of each ship is predicted by regression of the offset, target scale, and angle to realize the ship detection. Then, we adopt the spatial similarity of the ship center points to cluster the ship group, utilizing the Delaunay triangulation method to establish the topological graph structure of the ship group. Finally, we design a context-aware Dense Graph Convolutional Network (DGCN) with graph structure to achieve formation recognition. Experimental results on HRSD2016 and SGF datasets demonstrate that the proposed method can detect arbitrarily oriented ships and identify formations, attaining state-of-the-art performance.

Customizable 6 degrees of freedom grasping dataset and an interactive training method for graph convolutional network

The field of robotic grasping has seen significant progress with the development of deep learning and the creation of large-scale datasets like the Cornell Grasping Dataset (Jiang et al., 2011) and DexNet (Mahler et al., 2016). However, challenges persist due to the reliance on manually annotated datasets, limited by data scarcity, high costs, biases, and a lack of diversity in gripper types and three-dimensional information, hampering their effectiveness in real-world applications. To confront these issues, an innovative method was introduced for generating robotic grasping datasets in a simulated environment, eliminating the need for manual annotations. The method utilizes a highly realistic movement of the gripper, offering extensive customization options for a variety of gripper types. It also introduces detailed evaluation metrics specifically designed to assess different gripper designs, ensuring accurate and meaningful analysis of grasping efficacy. Further, it excels in simulating a wide range of industrial scenarios, significantly enhancing the dataset's diversity and applicability in real-world applications. In addition, an end-to-end grasping prediction network is introduced, which leverages advanced graph convolution techniques to predict optimal grasping points and orientations from point cloud. It also serves as an effective baseline for the proposed grasping dataset. Lastly, the authors propose a novel interactive training method for deep learning models driven by data generation, featuring real-time interaction between the model and the data generator with a rule-based strategy that optimizes the training workflow based on feedback. Experimental results demonstrate that the interactive training method enables models to achieve superior outcomes in a shorter timeframe compared to those trained using traditional methods.

An Adaptive Spatiotemporal Decouple Graph Convolutional Network Based Quality‐Related Fault Detection Method for Complex Industrial Processes

ABSTRACTWith the rapid development of industrial technology, industrial processes become increasingly complex, presenting characteristics of large‐scale and multi‐unit collaboration. However, most current fault detection methods focus on nonlinearity, dynamics, and other characteristics, while neglecting spatiotemporal information. To address this issue, an adaptive spatiotemporal decouple graph convolutional network based quality‐related fault detection method is proposed in this article. First, the temporal graph convolutional network and spatial graph convolutional network are combined organically in the form of joint training. Second, considering that fixed graph structures cannot reflect the dynamic relationships among nodes, we proposed an adaptive weighted mask mechanism to construct dynamic correlation graph embedded with priori knowledge. Multi‐attention mechanism is used to integrate spatiotemporal information, besides, we designed a decoupling layer to avoid information redundancy. Finally, the proposed spatiotemporal graph convolutional network is used to establish a regression model, the quality‐related latent variables are extracted by the decoupling layer, and the statistic is constructed based on Kullback–Leibler divergence. The effectiveness and feasibility of the proposed method are illustrated with the hot strip mill process and the Tennessee Eastman process.

Advanced Multi-Label Fire Scene Image Classification via BiFormer, Domain-Adversarial Network and GCN

Detecting wildfires presents significant challenges due to the presence of various potential targets in fire imagery, such as smoke, vehicles, and people. To address these challenges, we propose a novel multi-label classification model based on BiFormer’s feature extraction method, which constructs sparse region-indexing relations and performs feature extraction only in key regions, thereby facilitating more effective capture of flame characteristics. Additionally, we introduce a feature screening method based on a domain-adversarial neural network (DANN) to minimize misclassification by accurately determining feature domains. Furthermore, a feature discrimination method utilizing a Graph Convolutional Network (GCN) is proposed, enabling the model to capture label correlations more effectively and improve performance by constructing a label correlation matrix. This model enhances cross-domain generalization capability and improves recognition performance in fire scenarios. In the experimental phase, we developed a comprehensive dataset by integrating multiple fire-related public datasets, and conducted detailed comparison and ablation experiments. Results from the tenfold cross-validation demonstrate that the proposed model significantly improves recognition of multi-labeled images in fire scenarios. Compared with the baseline model, the mAP increased by 4.426%, CP by 4.14% and CF1 by 7.04%.

Efficient Multi-View Graph Convolutional Network with Self-Attention for Multi-Class Motor Imagery Decoding.

Research on electroencephalogram-based motor imagery (MI-EEG) can identify the limbs of subjects that generate motor imagination by decoding EEG signals, which is an important issue in the field of brain-computer interface (BCI). Existing deep-learning-based classification methods have not been able to entirely employ the topological information among brain regions, and thus, the classification performance needs further improving. In this paper, we propose a multi-view graph convolutional attention network (MGCANet) with residual learning structure for multi-class MI decoding. Specifically, we design a multi-view graph convolution spatial feature extraction method based on the topological relationship of brain regions to achieve more comprehensive information aggregation. During the modeling, we build an adaptive weight fusion (Awf) module to adaptively merge feature from different brain views to improve classification accuracy. In addition, the self-attention mechanism is introduced for feature selection to expand the receptive field of EEG signals to global dependence and enhance the expression of important features. The proposed model is experimentally evaluated on two public MI datasets and achieved a mean accuracy of 78.26% (BCIC IV 2a dataset) and 73.68% (OpenBMI dataset), which significantly outperforms representative comparative methods in classification accuracy. Comprehensive experiment results verify the effectiveness of our proposed method, which can provide novel perspectives for MI decoding.

Multi-sensor temporal-spatial graph network fusion empirical mode decomposition convolution for machine fault diagnosis

Multi-sensor time-series data at different locations contains not only temporal correlation information but also spatial correlation information which is treasure for machine fault diagnosis. Existing graph construction methods mainly apply different data analysis methods to connect nodes and edges. Few works, however, consider the location of the sensor itself and temporal correlation information of multi-sensor time-series data. To mine the relationship between spatial information and temporal information, the multi-sensor temporal-spatial graph is constructed in this paper. Hereinto, the different data points of multi-sensor are severed as different nodes which represents the spatial feature information. The temporal information is contained between different nodes of the same sensor. Moreover, an empirical mode decomposition graph convolution network (EGCN) is proposed to extract the feature. Specifically, the traditional graph convolution operator is changed to empirical mode decomposition which can decompose the input features into multiple intrinsic modal features to achieve adaptive feature extraction and improve the representation capability of the network. Finally, the different fault types can be classified by fully connected layers. Experiments from different test rigs demonstrate that the proposed method achieves a diagnostic accuracy exceeding 99 % under limited fault samples.

Efficient Visual-Aware Fashion Recommendation Using Compressed Node Features and Graph-Based Learning

In fashion e-commerce, predicting item compatibility using visual features remains a significant challenge. Current recommendation systems often struggle to incorporate high-dimensional visual data into graph-based learning models effectively. This limitation presents a substantial opportunity to enhance the precision and effectiveness of fashion recommendations. In this paper, we present the Visual-aware Graph Convolutional Network (VAGCN). This novel framework helps improve how visual features can be incorporated into graph-based learning systems for fashion item compatibility predictions. The VAGCN framework employs a deep-stacked autoencoder to convert the input image’s high-dimensional raw CNN visual features into more manageable low-dimensional representations. In addition to improving feature representation, the GCN can also reason more intelligently about predictions, which would not be possible without this compression. The GCN encoder processes nodes in the graph to capture structural and feature correlation. Following the GCN encoder, the refined embeddings are input to a multi-layer perceptron (MLP) to calculate compatibility scores. The approach extends to using neighborhood information only during the testing phase to help with training efficiency and generalizability in practical scenarios, a key characteristic of our model. By leveraging its ability to capture latent visual features and neighborhood-based learning, VAGCN thoroughly investigates item compatibility across various categories. This method significantly improves predictive accuracy, consistently outperforming existing benchmarks. These contributions tackle significant scalability and computational efficiency challenges, showcasing the potential transformation of recommendation systems through enhanced feature representation, paving the way for further innovations in the fashion domain.

Graph contrast learning for recommendation based on relational graph convolutional neural network

Current knowledge graph-based recommendation methods heavily rely on high-quality knowledge graphs, often falling short in effectively addressing issues such as the cold start problem and heterogeneous noise in user interactions. This leads to biases in user interest and popularity. To overcome these challenges, this paper introduces a novel recommendation approach termed Knowledge-enhanced Perceptive Graph Attention with Graph Contrastive Learning (KPA-GCL), which leverages relational graph convolutional neural networks. The proposed method optimizes the triplet embedding representation of entity-item interactions based on relationships between adjacent entities in a heterogeneous graph. Subsequently, a graph convolutional neural network is employed for enhanced aggregation. Similarity scores from a contrastive view serve as the selection criterion for high-quality embedded representations, facilitating the extraction of refined knowledge subgraphs. Multiple adaptive contrast-loss optimization functions are introduced by combining Bayesian Personalized Ranking (BPR) and hard negative sampling techniques. Comparative experiments are conducted with ten popular existing methods using real public datasets. Results indicate that the KPA-GCL method outperforms compared methods in all datasets based on Recall, NDCG, Precision, and Hit-ratio measures. Furthermore, in terms of mitigating cold start and noise, the KPA-GCL method surpasses other ten methods. This validates the reasonability and effectiveness of KPA-GCL in real-world datasets.

Superpixel perception graph neural network for intelligent defect detection of aero-engine blade

Aero-engine is the core component of aircraft and other spacecraft. The high-speed rotating blades provide power by sucking in air and fully combusting, and various defects will inevitably occur, threatening the operation safety of aero-engine. Therefore, regular inspections are essential for such a complex system. However, existing traditional technology which is borescope inspection is labor-intensive, time-consuming, and experience-dependent. To endow this technology with intelligence, a novel superpixel perception graph neural network (SPGNN) is proposed by utilizing a multi-stage graph convolutional network (MSGCN) for feature extraction and superpixel perception region proposal network (SPRPN) for region proposal. First, to capture complex and irregular textures, the images are transformed into a series of patches, to obtain their graph representations. Then, MSGCN composed of several GCN blocks extracts graph structure features and performs graph information processing at graph level. Last but not least, the SPRPN is proposed to generate perceptual bounding boxes by fusing graph representation features and superpixel perception features. Therefore, the proposed SPGNN always implements feature extraction and information transmission at the graph level in the whole SPGNN pipeline, to alleviate the reduction of receptive field and information loss. To verify the effectiveness of SPGNN, we construct a simulated blade dataset with 3000 images. A public aluminum dataset is also used to validate the performances of different methods. The experimental results demonstrate that the proposed SPGNN has superior performance compared with the state-of-the-art methods.

Multi-Granularity and Multi-Interest Contrast-Enhanced Hypergraph Convolutional Networks for Session Recommendation

Aiming at the problems of the incomplete recommendation and sparsity of session data in session recommendation, a new multi-granularity and multi-interest contrast-enhanced hypergraph convolutional network (MGMI-CEHCN) model for session recommendation is proposed. The session data are modeled as a heterogeneous hypergraph, the information is embedded by the hypergraph using two granularities of item price and category, and then the information between different granularities is fused, while the final item embedding is obtained through multi-layer convolution. Finally, an interest perceptron is used to detect multiple potential interests for each item, and a decentralized interest extraction network based on a gated recurrent unit (GRU) is used to integrate the user’s final interests and obtain a global session representation through a soft attention mechanism; a local session representation is generated with the help of a weighted line graph convolutional network. A further joint contrast enhancement strategy is used to maximize the mutual information between the global session representation and local session representation, to improve the recommendation performance. Experiments on several real datasets showed that the recommendation performance of the MGMI-CEHCN model outperformed the current mainstream models. On Cosmetics, the P@20 reached up to 55.25% and M@20 reached up to 38.26%, improvements of 3.06% and 3.09%, respectively; on Diginetica-buy, the P@20 reached up to 65.60% and M@20 reached up to 27.47%, improvements of 2.47% and 6.64%, respectively, which proved the validity of the model.

Collaborative energy management of interconnected regional integrated energy systems considering spatio-temporal characteristics

Multi-regional integrated energy systems (MRIES), which encompass renewable distributed generation (RDG) and electric vehicles (EV) dispersed across multiple regions, are pivotal for promoting low-carbon energy supplies, offering both economic and environmental benefits. However, the complexity introduced by diverse energy conversions and spatiotemporal coupling in MRIES poses significant challenges for energy management. This paper introduces a novel low-carbon energy management framework (EMF) for multi-regional IES, based on spatiotemporal correlations and considering three-dimensional (3D) integrated demand-side response (IDR). The proposed framework employs a novel Cross-Gated Spatio-Temporal Graph Convolutional Network (CGSTG) to achieve high-precision joint power forecasting separately for the RDG and multi-type loads. In the day-ahead stage, 3D IDR involving flexible loads, e.g., EVs, is considered with multi-regional IES sharing electricity and heat. By engaging in IDR across the temporal (horizontal), spatial (depth), and energy conversion (vertical) dimensions, each IES can enhance its coordinated scheduling capability and operational economy. During the intra-day stage, rolling optimization is performed based on the model predictive control (MPC) framework, incorporating a ladder-type carbon trading mechanism (CTM). Comparative experiments demonstrate that the proposed method can effectively reduce operational costs and carbon emissions with significant improvement of coordinated scheduling capability, leading to economic and environmental benefits of MRIES.