Graph Convolutional Layers Research Articles

A graph convolutional neural network model based on fused multi-subgraph as input and fused feature information as output

The graph convolution neural network (GCN)-based node classification model tackles the challenge of classifying nodes in graph data through learned feature representations. However, most existing graph neural networks primarily focus on the same type of edges, which might not accurately reflect the intricate real-world graph structure. This paper introduces a novel graph neural network model, MF-GCN, which integrates subgraphs with various edge types as input and combines feature information from each graph convolutional neural network layer to produce the final output. This model learns node feature representations by separately feeding subgraphs with different edge types into the graph convolutional layer. It then computes the weight vectors for fusing various edge type subgraphs based on the learned node features. Additionally, to efficiently extract feature information, the outputs of each graph convolution layer, without an activation function, are weighted and summed to obtain the final node features. This approach resolves the challenges of determining fusion weights and effectively extracting feature information during subgraph fusion. Experimental results show that the proposed model significantly improves performance on all three datasets, highlighting its effectiveness in node representation learning tasks.

Point cloud upsampling network based on pyramid pooling and self-attention mechanism

For the problems of irregular point cloud data and non-unique input size, a point-cloud upsampling network (PPSA-PU) based on Pyramid Pooling and Self-Attention mechanism is proposed. Multi-scale features are first extracted from the input point cloud using dense map convolution with different expansion rates, and the pyramid pooling module and self-attention mechanism are added. By applying the same pooling operation to regions of different sizes and numbers, the network is made to accept inputs of arbitrary size. Self-attention is used to weight the features to establish global correlation and improve the feature extraction capability. The features are then extended by the Self-Attention NodeShuffle (SA-NS) module, which uses a graph convolution layer to fuse neighbour information and learn new point features from the latent space. Finally, the coordinate reconstruction module is used to generate a dense up-sampled point cloud. Using the existing dataset to train and test the network and compare it with the existing up-sampling network, the experimental results show that PPSA-PU is better in terms of evaluation metrics and performance, and has better generalisation and robustness.

A novel label-aware global graph construction method and spiking-coded graph neural network for intelligent process fault diagnosis

Fault diagnosis plays a crucial role in ensuring the safety and efficiency of industrial processes. However, traditional techniques often face difficulties in handling large-scale data characterized by complex structures and relationships. To efficiently represent industrial data as graphs and develop a low-energy-cost feature extraction model, a novel label-aware global graph construction method and a spiking graph convolutional network (SGCN) are proposed in this study to achieve intelligent process fault diagnosis. The label-aware method enhances graph data representation by capturing intrinsic correlations and global features. The SGCN integrates graph convolutional layers with spiking encoding, enabling effective feature extraction while offering computational efficiency advantages. A weighted loss function is introduced to mitigate data imbalance issues. Experiments on the Tennessee Eastman process, the Three-phase Flow Facility, and the de-propanizer distillation process demonstrate SGCN’s superior performance over baseline models in various fault scenarios, while significantly reducing computational costs. The proposed method offers promising potential for reliable and efficient fault diagnosis in complex real-world industrial environments.

GCNPMDA: Human microbe-disease association prediction by hierarchical graph convolutional network with layer attention

Microorganisms play a crucial role in various physiological processes, including metabolism, immune defense, nutrition absorption, defense against cancer, and protection against pathogen colonization. Changes in microbial communities serve as potential biomarkers for diseases, offering significant insights into disease treatment and diagnosis. However, the association between microorganisms and diseases is still unclear, and more computational methods are needed to predict potential associations. In this paper, we introduce a novel computational model, the Graph Convolutional Network to Predict Microbe-Disease Associations (GCNPMDA), which employs layer attention mechanisms (see Figure 1). GCNPMDA integrates known microbe-disease associations, microbe–microbe similarities, and disease–disease similarities into a heterogeneous network. The model utilizes a Graph Convolutional Network (GCN) to learn embeddings for diseases and microbes. To enhance attribute information, microbe–microbe similarities are computed using Cosine similarity, Jaccard similarity, Gaussian kernel, and functional information, while disease–disease similarities are computed using Cosine similarity, Jaccard similarity, Gaussian kernel, and symptom information. Additionally, attention mechanisms are applied to combine embeddings from multiple graph convolution layers. The model’s predictive effectiveness is evaluated on Human Microbe-Disease Association Database (HMDAD). Leave-one-out cross-validation (LOOCV) was conducted. The Area Under ROC Curve (AUC) of LOOCV is 0.98. The 5-fold cross-validation (5-fold CV) on HMDAD yields average AUC of 0.98 ± 0.009. Furthermore, we carried out a case study of type 2 diabetes (T2D), inflammatory bowel disease (IBD), and rheumatoid arthritis. Based on existing literature evidence, it was confirmed that 6, 7, and 7 of the top-10 inferred microbes have established associations with T2D, IBD, and rheumatoid arthritis, respectively. GCNPMDA demonstrates potential efficacy in identifying disease-related microbes, offering a promising tool to uncover the intricate relationship between microorganisms and their human hosts.

AGNN: Alternating Graph-Regularized Neural Networks to Alleviate Over-Smoothing.

Graph convolutional network (GCN) with the powerful capacity to explore graph-structural data has gained noticeable success in recent years. Nonetheless, most of the existing GCN-based models suffer from the notorious over-smoothing issue, owing to which shallow networks are extensively adopted. This may be problematic for complex graph datasets because a deeper GCN should be beneficial to propagating information across remote neighbors. Recent works have devoted effort to addressing over-smoothing problems, including establishing residual connection structure or fusing predictions from multilayer models. Because of the indistinguishable embeddings from deep layers, it is reasonable to generate more reliable predictions before conducting the combination of outputs from various layers. In light of this, we propose an alternating graph-regularized neural network (AGNN) composed of graph convolutional layer (GCL) and graph embedding layer (GEL). GEL is derived from the graph-regularized optimization containing Laplacian embedding term, which can alleviate the over-smoothing problem by periodic projection from the low-order feature space onto the high-order space. With more distinguishable features of distinct layers, an improved Adaboost strategy is utilized to aggregate outputs from each layer, which explores integrated embeddings of multi-hop neighbors. The proposed model is evaluated via a large number of experiments including performance comparison with some multilayer or multi-order graph neural networks, which reveals the superior performance improvement of AGNN compared with the state-of-the-art models.

D-MHGCN: An End-to-End Individual Behavioral Prediction Model Using Dual Multi-Hop Graph Convolutional Network.

Predicting individual behavior is a crucial area of research in neuroscience. Graph Neural Networks (GNNs), as powerful tools for extracting graph-structured features, are increasingly being utilized in various functional connectivity (FC) based behavioral prediction tasks. However, current predictive models primarily focus on enhancing GNNs' ability to extract features from FC networks while neglecting the importance of upstream individual network construction quality. This oversight results in constructed functional networks that fail to adequately represent individual behavioral capacity, thereby affecting the subsequent prediction accuracy. To address this issue, we proposed a new GNN-based behavioral prediction framework, named Dual Multi-Hop Graph Convolutional Network (D-MHGCN). Through the joint training of two GCNs, this framework integrates individual functional network construction and behavioral prediction into a unified optimization model. It allows the model to dynamically adjust the individual functional cortical parcellation according to the downstream tasks, thus creating task-aware, individual-specific FCNs that largely enhance its ability to predict behavior scores. Additionally, we employed multi-hop graph convolution layers instead of traditional single-hop methods in GCN to capture complex hierarchical connectivity patterns in brain networks. Our experimental evaluations, conducted on the large, public Human Connectome Project dataset, demonstrate that our proposed method outperforms existing methods in various behavioral prediction tasks. Moreover, it produces more functionally homogeneous cortical parcellation, showcasing its practical utility and effectiveness. Our work not only enhances the accuracy of individual behavioral prediction but also provides deeper insights into the neural mechanisms underlying individual differences in behavior.

Molecular Classification with Graph ConvolutionalNetworks: Exploring the MUTAG Dataset for Mutagenicity Prediction

This paper presents the implementation of a Graph Convolutional Network (GCN) for the classification of chemical compounds using the MUTAG dataset, which consists of 188 ni- troaromatic compounds labeled according to their mutagenicity. The GCN model leverages the inherent graph structure of molec-ular data to capture and learn from the relationships between atoms and bonds, represented as nodes and edges, respectively. By utilizing three graph convolutional layers followed by a global mean pooling layer, the model effectively aggregates node features to generate meaningful graph-level representations. The model was trained using the Adam optimizer with a learning rate of 0.01, and cross-entropy loss was employed to supervise the classification task. The results demonstrate the efficacy of GCNs in graph classification tasks, with the model achieving a training accuracy of 79.33% and a test accuracy of 76.32%. This study highlights the potential of GCNs in cheminformatics and other domains where graph-structured data is prevalent, paving the way for further exploration and application of advanced graph neural networks in similar tasks.

Sub-sampling graph neural networks for genomic prediction of quantitative phenotypes.

In genomics, use of deep learning (DL) is rapidly growing and DL has successfully demonstrated its ability to uncover complex relationships in large biological and biomedical data sets. With the development of high-throughput sequencing techniques, genomic markers can now be allocated to large sections of a genome. By analysing allele sharing between individuals, one may calculate realized genomic relationships from single nucleotide polymorphisms (SNPs) data rather than relying on known pedigree relationships under polygenic model. The traditional approaches in genome-wide prediction (GWP) of quantitative phenotypes utilise genomic relationships in fixed global covariance modelling, possibly with some non-linear kernel mapping (for example Gaussian processes). On the other hand, the DL approaches proposed so far for GWP fail to take into account the non-Euclidean graph structure of relationships between individuals over several generations. In this paper, we propose one global convolutional neural network (GCN) and one local sub-sampling architecture (GCN-RS) that are specifically designed to perform regression analysis based on genomic relationship information. A GCN is tailored to non-Euclidean spaces and consists of several layers of graph convolutions. The GCN-RS architecture is designed to further improve the GCN's performance by sub-sampling the graph to reduce the dimensionality of the input data. Through these graph convolutional layers, the GCN maps input genomic markers to their quantitative phenotype values. The graphs are constructed using an iterative nearest neighbour approach. Comparisons show that the GCN-RS outperforms the popular Genomic Best Linear Unbiased Predictor (GBLUP) method on one simulated and three real data sets from wheat, mice and pig with a predictive improvement of 4.4% to 49.4% in terms of test mean squared error (MSE). This indicates that GCN-RS is a promising tool for genomic predictions in plants and animals. Furthermore, GCN-RS is computationally efficient, making it a viable option for large-scale applications.

SAGE-GSAN: A graph-based method for estimating urban taxi CO emissions using street view images

Accurately predicting the carbon emissions of urban street vehicles is a current challenge in the field of urban transportation. This study proposed a new SAGE-GSAN model (Graph SAmple and aggreGatE - Graph Spatial Attention Network) to solve this problem. It combines graph neural networks with streetscape features and road network structure to predict the traffic carbon monoxide emissions at the street level. The method takes the street view images, the 5075 roads network structure in Wuhan and 19 street visual elements as the input features, and the carbon monoxide emissions obtained from the driving trajectories as the prediction data. The method achieved an experimental accuracy of 81.4% in predicting carbon monoxide emissions from street cabs. This study also compares the prediction results of traditional neural networks and analyzes the effects of different street-level features and graph convolution layers on the prediction accuracy. The results of this study show that the graph neural network and attention mechanism techniques could solve the fine-grained carbon emission prediction problem at the urban street level effectively. The model code is shared at the: https://github.com/zou9229/CO_Predict_Code.

GCN-Transformer-Based Spatio-Temporal Load Forecasting for EV Battery Swapping Stations under Differential Couplings

To address the challenge of power absorption in grids with high renewable energy integration, electric vehicle battery swapping stations (EVBSSs) serve as critically important flexible resources. Current research on load forecasting for EVBSSs primarily employs Transformer models, which have increasingly shown a lack of adaptability to the rapid growth in scale and complexity. This paper proposes a novel data-driven forecasting model that combines the geographical feature extraction capability of graph convolutional networks (GCNs) with the multitask learning capability of Transformers. The GCN-Transformer model first leverages Spearman’s rank correlation to create a multinode feature set encompassing date, weather, and historical load data. It then employs data-adaptive graph generation for dynamic spatio-temporal graph construction and graph convolutional layers for spatial aggregation tailored to each node. Unique swapping patterns are identified through node-adaptive parameter learning, while the temporal dynamics of multidimensional features are managed by the Transformer’s components. Numerical results demonstrate enhanced accuracy and efficiency in load forecasting for multiple and widely distributed EVBSSs.

EQGraphNet: Advancing single-station earthquake magnitude estimation via deep graph networks with residual connections

Magnitude estimation is a critical task in seismology, and conventional methods usually require dense seismic station arrays to provide data with sufficient spatiotemporal distribution. In this context, we propose the Earthquake Graph Network (EQGraphNet) to enhance the performance of single-station magnitude estimation. The backbone of the proposed model consists of eleven convolutional neural network layers and ten RCGL modules, where a RCGL combines a Residual Connection and a Graph convolutional Layer capable of mitigating the over-smoothing problem and simultaneously extracting temporal features of seismic signals. Our work uses the STanford EArthquake Dataset for model training and performance testing. Compared with three existing deep learning models, EQGraphNet demonstrates improved accuracy for both local magnitude and duration magnitude scales. To evaluate the robustness, we add natural background noise to the model input and find that EQGraphNet achieves the best results, particularly for signals with lower signal-to-noise ratios. Additionally, by replacing various network components and comparing their estimation performances, we illustrate the contribution of each part of EQGraphNet, validating the rationality of our approach. We also demonstrate the generalization capability of our model across different earthquakes occurring environments, achieving mean errors of ±0.1 units. Furthermore, by demonstrating the effectiveness of deeper architectures, this work encourages further exploration of deeper GNN models for both multi-station and single-station magnitude estimation.

Rotating machinery fault diagnosis based on one-dimensional convolutional neural network and modified multi-scale graph convolutional network under limited labeled data

Current rotating machinery fault diagnosis methods are primarily based on deep learning methods, which necessitate a large amount of data for training to achieve better results. However, it is quite challenging to collect and mark fault data in industry, limiting the use of mainstream methods. In this paper, a new framework for rotating machinery fault diagnosis is proposed that combines one-dimensional convolutional neural network and modified multi-scale graph convolutional network. Firstly, the category information of samples and the correlation between samples are used to build the graph-structured data. Secondly, by cascading standard graph convolutional layers, a modified multi-scale graph convolutional network is proposed, which makes the model better at extracting features at different scales. Finally, the one-dimensional convolutional neural network and the modified multi-scale graph convolutional network are combined, and an optimized training strategy is introduced to address the problem of model training overfitting with limited labeled data. The proposed method is shown to be superior for rotating machinery fault diagnosis with limited labeled data by applying the proposed framework to different datasets and comparing it to various diagnostic models.

Rolling Bearing Fault Diagnosis in Agricultural Machinery Based on Multi-Source Locally Adaptive Graph Convolution

Maintaining agricultural machinery is crucial for efficient mechanized farming. Specifically, diagnosing faults in rolling bearings, which are essential rotating components, is of significant importance. Domain-adaptive technology often addresses the challenge of limited labeled data from a single source domain. However, information transfer can sometimes fall short in providing adequate relevant details for supporting target diagnosis tasks, leading to poor recognition performance. This paper introduces a novel fault diagnosis model based on a multi-source locally adaptive graph convolution network to diagnose rolling bearing faults in agricultural machinery. The model initially employs an overlapping sampling method to enhance sample data. Recognizing that two-dimensional time–frequency signals possess richer spatial characteristics in neural networks, wavelet transform is used to convert time series samples into time–frequency graph samples before feeding them into the feature network. This approach constructs a sample data pair from both source and target domains. Furthermore, a feature extraction network is developed by integrating the strengths of deep residual networks and graph convolutional networks, enabling the model to better learn invariant features across domains. The locally adaptive method aids the model in more effectively aligning features from the source and target domains. The model incorporates a Softmax layer as the bearing state classifier, which is set up after the graph convolutional network layer, and outputs bearing state recognition results upon reaching a set number of iterations. The proposed method’s effectiveness was validated using a bearing dataset from Jiangnan University. For three different groups of bearing fault diagnosis tasks under varying working conditions, the proposed method achieved recognition accuracies above 99%, with an improvement of 0.30%-4.33% compared to single-source domain diagnosis models. Comparative results indicate that the proposed method can effectively identify bearing states even without target domain labels, showcasing its practical engineering application value.

Hand-aware graph convolution network for skeleton-based sign language recognition

Skeleton-based sign language recognition (SLR) is a challenging research area mainly due to the fast and complex hand movement. Currently, graph convolution networks (GCNs) have been employed in skeleton-based SLR and achieved remarkable performance. However, existing GCN-based SLR methods suffer from a lack of explicit attention to hand topology which plays an important role in the sign language representation. To address this issue, we propose a novel hand-aware graph convolution network (HA-GCN) to focus on hand topological relationships of skeleton graph. Specifically, a hand-aware graph convolution layer is designed to capture both global body and local hand information, in which two sub-graphs are defined and incorporated to represent hand topology information. In addition, in order to eliminate the over-fitting problem, an adaptive DropGraph is designed in construction of hand-aware graph convolution block to remove the spatial and temporal redundancy in the sign language representation. With the aim to further improve the performance, the joints information, bones, together with their motion information are simultaneously modeled in a multi-stream framework. Extensive experiments on the two open-source datasets, AUTSL and INCLUDE, demonstrate that our proposed algorithm outperforms the state-of-the-art with a significant margin. Our code is available at https://github.com/snorlaxse/HA-SLR-GCN.

High-frequency and low-frequency dual-channel graph attention network

Most existing graph convolution layers use learnable or fixed weights to sum up neighbor features to aggregate neighbor information. Since the attention values are always positive, these graph convolution layers perform as low-pass filters, which may result in their poor performance on heterophilic graphs. In this paper, two graph convolutional layers are proposed, HLGAT and NGAT. NGAT is a convolution network using only negative attention values, which only make the aggregation of high-frequency information of neighbor nodes. HLGAT makes the aggregation of low-frequency and high-frequency information by two channels, respectively, and fuses two outputs by using a learnable way. On node-classification task, both NGAT and HLGAT offer significant performance improvement compared to existing methods. The results clearly show that: (1) High-frequency information of neighborhoods plays a decisive role in heterophilic graphs. (2) The aggregation of low-frequency and high-frequency information of neighbor nodes can significantly improve the performance on heterophilic graphs.

Quantitative Stock Selection Model Using Graph Learning and a Spatial–Temporal Encoder

In the rapidly evolving domain of finance, quantitative stock selection strategies have gained prominence, driven by the pursuit of maximizing returns while mitigating risks through sophisticated data analysis and algorithmic models. Yet, prevailing models frequently neglect the fluid dynamics of asset relationships and market shifts, a gap that undermines their predictive and risk management efficacy. This oversight renders them vulnerable to market volatility, adversely affecting investment decision quality and return consistency. Addressing this critical gap, our study proposes the Graph Learning Spatial–Temporal Encoder Network (GL-STN), a pioneering model that seamlessly integrates graph theory and spatial–temporal encoding to navigate the intricacies and variabilities of financial markets. By harnessing the inherent structural knowledge of stock markets, the GL-STN model adeptly captures the nonlinear interactions and temporal shifts among assets. Our innovative approach amalgamates graph convolutional layers, attention mechanisms, and long short-term memory (LSTM) networks, offering a comprehensive analysis of spatial–temporal data features. This integration not only deciphers complex stock market interdependencies but also accentuates crucial market insights, enabling the model to forecast market trends with heightened precision. Rigorous evaluations across diverse market boards—Main Board, SME Board, STAR Market, and ChiNext—underscore the GL-STN model’s exceptional ability to withstand market turbulence and enhance profitability, affirming its substantial utility in quantitative stock selection.

Learning intrinsic shape representations via spectral mesh convolutions

We introduce spectral-based convolutional operators embedded within Generalized Graph Neural Networks (G-GNNs). These operators enable deep learning on graphs through a learnable, energy-driven evolution process. This approach empowers us to impose specific properties on the graph convolutional kernel directly derived from the corresponding variational formulations. Our model incorporates both parameterized and non-parameterized graph Laplacian-based energies within the generalized graph convolutional layer to address features like smoothness, sharpness, and compact support. By making appropriate choices within our G-GNN framework, we pave the way for designing novel paradigms for 3D shape representation, reconstruction, and processing, while also enabling effective feature embeddings for intrinsic neural fields.

HyperGCN – a multi-layer multi-exit graph neural network to enhance hyperspectral image classification

ABSTRACT Graph neural networks (GNNs) have recently garnered significant attention due to their exceptional performance across various applications, including hyperspectral (HS) image classification. However, most existing GNN-based models for HS image classification are limited depth models and often suffer from performance degradation as model depth increases. This study introduces HyperGCN, an exclusive GNN-based model designed with multiple graph convolutional layers to exploit the rich spectral information inherent in HS images, thereby enhancing classification performance. To address performance degradation, HyperGCN incorporates techniques resistant to oversmoothing into its architecture. Additionally, multiple-side exit branches are integrated into the intermediate layers of HyperGCN, enabling dynamic management of the complexity of HS images. Less complex HS images are processed by fewer layers, exiting early via attached branches, while more complex images traverse multiple layers until reaching the final output layer. Extensive experiments on four benchmark HS datasets (Indian Pines, Pavia University, Salinas, and Botswana) demonstrate HyperGCN’s superior performance over basic GNN-based models. Notably, HyperGCN outperforms or performs comparably to the CNN-GNN combined model in classifying HS images. Furthermore, the superior performance of multi-exit HyperGCN over its single-exit counterpart emphasizes the effectiveness of incorporating side exit branches in GNN-based HS image classification. Compared to state-of-the-art models, multi-exit HyperGCN demonstrates competitive performance, highlighting its effectiveness in handling complex spectral information in HS images while maintaining an acceptable balance between accuracy and computational efficiency.

Bedding-parallel fracture density prediction using graph convolutional network in continental shale oil reservoirs: A case study in Mahu Sag, Junggar basin, China

The natural fractures, e.g., the bedding-parallel fractures (BPF), have a significant impact on the storage space and horizontal permeability of continental shale oil reservoirs. However, the conventional logging response of BPF is complex. To solve the problem of BPF density prediction, we propose the Edge Generation Weighted Graph Convolutional Network (EG-WGCN) method, which is an improved Graph Convolutional Network (GCN). The new method integrates the relationship information between the fractures and lithology into the edge generation method, and adds the generated edge weight information into the message passing process, thereby effectively improving the accuracy of fracture density prediction. The prediction process is divided into three steps: first, using conventional logging sampling points as vertices, establish vertex sets for each lithology and single lithologic layer. Second, based on the depth sequence of a single lithologic layer and the Euclidean distance of the same lithologic vertex sets, the connection between vertices is established and overlapping edges generated by the above methods are given higher weights. Third, the vertices in the constructed graph are classified through graph convolutional layers, which are integrated into the edge weight information. This method is applied to the fracture density prediction of continental shale oil reservoirs in the Mahu Sag of the Junggar Basin, western China. The results show that the accuracy of our proposed EG-WGCN method in testing data is 95.13%. Compared to the GCN (boundless) without using the edge generation method and EG-GCN with the edge generation method but without the weighted aggregation mechanism, model accuracy is improved by 14.48% and 1.22%, respectively. In summary, this method proves that if a proper ML model is used, conventional logging data can become valuable for predicting BPF density with high accuracy even when cores are missing.

Phase-wise attention GCN for recommendation denoising

In recent years, Graph Convolution Network (GCN) has been widely applied in recommendation systems. Compared to traditional collaborative filtering methods that only aggregate information of neighboring nodes, GCN-based recommendation systems can aggregate information of high-order neighborhoods of a node to achieve better performance. However, GCN-based recommendation systems have faced the over-smoothing problem. The over-smoothing problem causes the performance of recommender systems first to rise and then decrease as the number of stacked layers increases. Previous GCN-based recommender systems have provided a few solutions to over-smoothing. We believe that there are still two problems: (1) most models focus on aggregating diverse information, with less consideration of filtering noise; (2) most models ignore the fact that there are differences in different stages of the process of aggregating information using GCN. We introduce a Phase-wise Attention mechanism to address these issues and propose a Phase-wise Attention GCN (PAGCN) recommendation model. Considering the different characteristics of different stages during the aggregation process of the GCN-based recommendation systems, our model uses different information aggregation methods in different graph convolution layers and adopts targeted aggregation methods. In this way, high-order neighborhood information can be more controlled to improve the performance and effectiveness of our model. The experimental results on real-world datasets show that our model outperforms various baselines, demonstrating the reasonableness of our method.