Deep Graph Convolutional Network Research Articles

Spatiotemporal smoothing aggregation enhanced multi-scale residual deep graph convolutional networks for skeleton-based gait recognition

Spatiotemporal smoothing aggregation enhanced multi-scale residual deep graph convolutional networks for skeleton-based gait recognition

DeepGCN based on variable multi‐graph and multimodal data for ASD diagnosis

AbstractDiagnosing individuals with autism spectrum disorder (ASD) accurately faces great challenges in clinical practice, primarily due to the data's high heterogeneity and limited sample size. To tackle this issue, the authors constructed a deep graph convolutional network (GCN) based on variable multi‐graph and multimodal data (VMM‐DGCN) for ASD diagnosis. Firstly, the functional connectivity matrix was constructed to extract primary features. Then, the authors constructed a variable multi‐graph construction strategy to capture the multi‐scale feature representations of each subject by utilising convolutional filters with varying kernel sizes. Furthermore, the authors brought the non‐imaging information into the feature representation at each scale and constructed multiple population graphs based on multimodal data by fully considering the correlation between subjects. After extracting the deeper features of population graphs using the deep GCN(DeepGCN), the authors fused the node features of multiple subgraphs to perform node classification tasks for typical control and ASD patients. The proposed algorithm was evaluated on the Autism Brain Imaging Data Exchange I (ABIDE I) dataset, achieving an accuracy of 91.62% and an area under the curve value of 95.74%. These results demonstrated its outstanding performance compared to other ASD diagnostic algorithms.

Heterogeneous text graph for comprehensive multilingual sentiment analysis: capturing short- and long-distance semantics.

Multilingual sentiment analysis (MSA) involves the task of comprehending people's opinions, sentiments, and emotions in multilingual written texts. This task has garnered considerable attention due to its importance in extracting insights for decision-making across diverse fields such as marketing, finance, and politics. Several studies have explored MSA using deep learning methods. Nonetheless, a majority of these studies depend on sequential-based approaches, which focus on capturing short-distance semantics within adjacent word sequences, but they overlook long-distance semantics, which can provide more profound insights for analysis. In this work, we propose an approach for multilingual sentiment analysis, namely MSA-GCN, leveraging a graph convolutional network to effectively capture both short- and long-distance semantics. MSA-GCN involves the comprehensive modeling of the multilingual sentiment analysis corpus through a unified heterogeneous text graph. Subsequently, a slightly deep graph convolutional network is employed to acquire predictive representations for all nodes by encouraging the transfer learning across languages. Extensive experiments are carried out on various language combinations using different benchmark datasets to assess the efficiency of the proposed approach. These datasets include Multilingual Amazon Reviews Corpus (MARC), Internet Movie Database (IMDB), Allociné, and Muchocine. The achieved results reveal that MSA-GCN significantly outperformed all baseline models in almost all datasets with a p-value < 0.05 based on student t-test. In addition, such approach shows prominent results in a variety of language combinations, revealing the robustness of the approach against language variation.

A Very Deep Graph Convolutional Network for 13C NMR Chemical Shift Calculations with Density Functional Theory Level Performance for Structure Assignment.

Nuclear magnetic resonance (NMR) chemical shift calculations are powerful tools for structure elucidation and have been extensively employed in both natural product and synthetic chemistry. However, density functional theory (DFT) NMR chemical shift calculations are usually time-consuming, while fast data-driven methods often lack reliability, making it challenging to apply them to computationally intensive tasks with a high requirement on quality. Herein, we have constructed a 54-layer-deep graph convolutional network for 13C NMR chemical shift calculations, which achieved high accuracy with low time-cost and performed competitively with DFT NMR chemical shift calculations on structure assignment benchmarks. Our model utilizes a semiempirical method, GFN2-xTB, and is compatible with a broad variety of organic systems, including those composed of hundreds of atoms or elements ranging from H to Rn. We used this model to resolve the controversial J/K ring junction problem of maitotoxin, which is the largest whole molecule assigned by NMR calculations to date. This model has been developed into user-friendly software, providing a useful tool for routine rapid structure validation and assignation as well as a new approach to elucidate the large structures that were previously unsuitable for NMR calculations.

Separable Deep Graph Convolutional Network Integrated With CNN and Prototype Learning for Hyperspectral Image Classification

Separable Deep Graph Convolutional Network Integrated With CNN and Prototype Learning for Hyperspectral Image Classification

Node-Smoothness Based Adaptive Initial Residual Deep Graph Convolutional Network

Node-Smoothness Based Adaptive Initial Residual Deep Graph Convolutional Network

SkipNode: On Alleviating Performance Degradation for Deep Graph Convolutional Networks

SkipNode: On Alleviating Performance Degradation for Deep Graph Convolutional Networks

Structure-Aware DropEdge Toward Deep Graph Convolutional Networks.

It has been discovered that graph convolutional networks (GCNs) encounter a remarkable drop in performance when multiple layers are piled up. The main factor that accounts for why deep GCNs fail lies in oversmoothing, which isolates the network output from the input with the increase of network depth, weakening expressivity and trainability. In this article, we start by investigating refined measures upon DropEdge-an existing simple yet effective technique to relieve oversmoothing. We term our method as DropEdge ++ for its two structure-aware samplers in contrast to DropEdge: layer-dependent (LD) sampler and feature-dependent (FD) sampler. Regarding the LD sampler, we interestingly find that increasingly sampling edges from the bottom layer yields superior performance than the decreasing counterpart as well as DropEdge. We theoretically reveal this phenomenon with mean-edge-number (MEN), a metric closely related to oversmoothing. For the FD sampler, we associate the edge sampling probability with the feature similarity of node pairs and prove that it further correlates the convergence subspace of the output layer with the input features. Extensive experiments on several node classification benchmarks, including both full-and semi-supervised tasks, illustrate the efficacy of DropEdge ++ and its compatibility with a variety of backbones by achieving generally better performance over DropEdge and the no-drop version.

Intelligent Compound Fault Diagnosis of Roller Bearings Based on Deep Graph Convolutional Network.

The high correlation between rolling bearing composite faults and single fault samples is prone to misclassification. Therefore, this paper proposes a rolling bearing composite fault diagnosis method based on a deep graph convolutional network. First, the acquired raw vibration signals are pre-processed and divided into sub-samples. Secondly, a number of sub-samples in different health states are constructed as graph-structured data, divided into a training set and a test set. Finally, the training set is used as input to a deep graph convolutional neural network (DGCN) model, which is trained to determine the optimal structure and parameters of the network. A test set verifies the feasibility and effectiveness of the network. The experimental result shows that the DGCN can effectively identify compound faults in rolling bearings, which provides a new approach for the identification of compound faults in bearings.

Deep graph convolutional network for small-molecule retention time prediction

The retention time (RT) is a crucial source of data for liquid chromatography-mass spectrometry (LCMS). A model that can accurately predict the RT for each molecule would empower filtering candidates with similar spectra but differing RT in LCMS-based molecule identification. Recent research shows that graph neural networks (GNNs) outperform traditional machine learning algorithms in RT prediction. However, all of these models use relatively shallow GNNs. This study for the first time investigates how depth affects GNNs' performance on RT prediction. The results demonstrate that a notable improvement can be achieved by pushing the depth of GNNs to 16 layers by the adoption of residual connection. Additionally, we also find that graph convolutional network (GCN) model benefits from the edge information. The developed deep graph convolutional network, DeepGCN-RT, significantly outperforms the previous state-of-the-art method and achieves the lowest mean absolute percentage error (MAPE) of 3.3% and the lowest mean absolute error (MAE) of 26.55 s on the SMRT test set. We also finetune DeepGCN-RT on seven datasets with various chromatographic conditions. The mean MAE of the seven datasets largely decreases 30% compared to previous state-of-the-art method. On the RIKEN-PlaSMA dataset, we also test the effectiveness of DeepGCN-RT in assisting molecular structure identification. By 30% lessening the number of potential structures, DeepGCN-RT is able to improve top-1 accuracy by about 11%.

Mal2GCN: a robust malware detection approach using deep graph convolutional networks with non-negative weights

Mal2GCN: a robust malware detection approach using deep graph convolutional networks with non-negative weights

A deep graph convolutional network model of NOx emission prediction for coal‐fired boiler

AbstractTo deal with environmental problems caused by NOx production in thermal plants, it is imperative to establish a reliable model to predict NOx concentration in the combustion process. NOx formation in a coal‐fired boiler is complex, and many variables affect NOx emissions. The effective information fusion of these variables can improve the accuracy of NOx concentration prediction. However, the existing NOx prediction algorithms based on thermal parameters rarely consider the mechanical knowledge of the boiler operation, and it is not easy to incorporate the topological information of production into modelling. Therefore, a graph convolutional network is proposed for NOx emission prediction. First, the key variables affecting NOx generation are selected according to the knowledge and the random forest‐based variable importance. Then, the model structure is designed by exploring the topological information among thermal variables to capture the complex spatial dependence. The model inputs are constructed by coding different operation variables, and the adjacency matrix is generated according to the correlation information between variables, which can fuse data information and reduce redundancy. On this basis, the prediction model of NOx concentration is established. Historical data from a 660 MW coal‐fired boiler are used in the experiment. The prediction results show that the proposed model can effectively fuse the information of characteristic variables and fully exploit the non‐linear mapping relationship between process variables and NOx emission. When compared with three typical models in NOx modelling, the proposed model has better performance with a determination coefficient of 0.906.

To see further: Knowledge graph-aware deep graph convolutional network for recommender systems

Applying a graph convolutional network (GCN) or its variants to user-item interaction graphs is one of the most commonly used approaches for learning the representation of users and items in modern recommender systems. However, these models perform poorly in recommending high-order items due to the over-smoothing when the GCN deepens and the lack of connectivity between the cold-start items and the target user. To improve the ability to recommend higher-order items, we propose our Knowledge graph-aware Deep Graph Convolutional Network with Initial Residual Connection (KDGCN-IC) and Knowledge graph-aware Deep Graph Convolutional Network with Dense Connection (KDGCN-DC) methods. First, we introduce an item knowledge graph to rebuild the connectivity between nodes in the user-item graph, especially the users and the cold-start items. Then, we present our methods for improving the information flow by reusing features to alleviate the over-smoothing issue to deepen the GCN to capture the higher-order collaborative signal. More specifically, KDGCN-IC reuses the initial feature information, and KDGCN-DC reuses the feature information of each layer. Extensive experiments demonstrated that our models were able to achieve substantial improvement over state-of-the-art models in so far as their recommendation accuracy and higher-order recommendations.

Automatic Calcification Morphology and Distribution Classification for Breast Mammograms with Multi-task Graph Convolutional Neural Network.

The morphology and distribution of microcalcifications are the most important descriptors for radiologists to diagnose breast cancer based on mammograms. However, it is very challenging and time-consuming for radiologists to characterize these descriptors manually, and there also lacks of effective and automatic solutions for this problem. We observed that the distribution and morphology descriptors are determined by the radiologists based on the spatial and visual relationships among calcifications. Thus, we hypothesize that this information can be effectively modelled by learning a relationship-aware representation using graph convolutional networks (GCNs). In this study, we propose a multi-task deep GCN method for automatic characterization of both the morphology and distribution of microcalcifications in mammograms. Our proposed method transforms morphology and distribution characterization into node and graph classification problem and learns the representations concurrently. We trained and validated the proposed method in an in-house dataset and public DDSM dataset with 195 and 583 cases,respectively. The proposed method reaches good and stable results with distribution AUC at 0.812 ±0.043 and 0.873 ±0.019, morphology AUC at 0.663 ±0.016 and 0.700 ±0.044 for both in-house and public datasets. In both datasets, our proposed method demonstrates statistically significant improvements compared to the baseline models. The performance improvements brought by our proposed multi-task mechanism can be attributed to the association between the distribution and morphology of calcifications in mammograms, which is interpretable using graphical visualizations and consistent with the definitions of descriptors in the standard BI-RADS guideline. In short, we explore, for the first time, the application of GCNs in microcalcification characterization that suggests the potential of using graph learning for more robust understanding of medical images.

Research on 3D advertising placement based on virtual reality simulation

AbstractVirtual reality based on computer 3D technology has brought revolutionary changes in advertising creativity. To enhance the economic value of advertising itself, it is necessary to provide personalized and precise advertising placement for users from a massive advertising database. A recommendation system based on deep graph convolutional networks is a personalized system that can recommend based on their attributes and historical records. However, complex models are required to learn higher‐order feature information, and the mining of user interaction behavior is insufficient. This article proposes a graph convolutional network with multi‐head attention embedding for 3D advertising placement. We construct advertising graph data based on user behavior and multi‐feature embedding mapping. To exploit higher‐order features, a multi‐head attention mechanism is embedded in the graph convolutional network. The experimental results show that the proposed model achieves better performance compared to other depth models on Criteo and Avazu datasets. We will deploy the entire model on the server to achieve precise advertising placement.

GDCL-NcDA: identifying non-coding RNA-disease associations via contrastive learning between deep graph learning and deep matrix factorization

Non-coding RNAs (ncRNAs) draw much attention from studies widely in recent years because they play vital roles in life activities. As a good complement to wet experiment methods, computational prediction methods can greatly save experimental costs. However, high false-negative data and insufficient use of multi-source information can affect the performance of computational prediction methods. Furthermore, many computational methods do not have good robustness and generalization on different datasets. In this work, we propose an effective end-to-end computing framework, called GDCL-NcDA, of deep graph learning and deep matrix factorization (DMF) with contrastive learning, which identifies the latent ncRNA-disease association on diverse multi-source heterogeneous networks (MHNs). The diverse MHNs include different similarity networks and proven associations among ncRNAs (miRNAs, circRNAs, and lncRNAs), genes, and diseases. Firstly, GDCL-NcDA employs deep graph convolutional network and multiple attention mechanisms to adaptively integrate multi-source of MHNs and reconstruct the ncRNA-disease association graph. Then, GDCL-NcDA utilizes DMF to predict the latent disease-associated ncRNAs based on the reconstructed graphs to reduce the impact of the false-negatives from the original associations. Finally, GDCL-NcDA uses contrastive learning (CL) to generate a contrastive loss on the reconstructed graphs and the predicted graphs to improve the generalization and robustness of our GDCL-NcDA framework. The experimental results show that GDCL-NcDA outperforms highly related computational methods. Moreover, case studies demonstrate the effectiveness of GDCL-NcDA in identifying the associations among diversiform ncRNAs and diseases.

DRGCN: Dynamic Evolving Initial Residual for Deep Graph Convolutional Networks

Graph convolutional networks (GCNs) have been proved to be very practical to handle various graph-related tasks. It has attracted considerable research interest to study deep GCNs, due to their potential superior performance compared with shallow ones. However, simply increasing network depth will, on the contrary, hurt the performance due to the over-smoothing problem. Adding residual connection is proved to be effective for learning deep convolutional neural networks (deep CNNs), it is not trivial when applied to deep GCNs. Recent works proposed an initial residual mechanism that did alleviate the over-smoothing problem in deep GCNs. However, according to our study, their algorithms are quite sensitive to different datasets. In their setting, the personalization (dynamic) and correlation (evolving) of how residual applies are ignored. To this end, we propose a novel model called Dynamic evolving initial Residual Graph Convolutional Network (DRGCN). Firstly, we use a dynamic block for each node to adaptively fetch information from the initial representation. Secondly, we use an evolving block to model the residual evolving pattern between layers. Our experimental results show that our model effectively relieves the problem of over-smoothing in deep GCNs and outperforms the state-of-the-art (SOTA) methods on various benchmark datasets. Moreover, we develop a mini-batch version of DRGCN which can be applied to large-scale data. Coupling with several fair training techniques, our model reaches new SOTA results on the large-scale ogbn-arxiv dataset of Open Graph Benchmark (OGB). Our reproducible code is available on GitHub.

Orders Are Unwanted: Dynamic Deep Graph Convolutional Network for Personality Detection

Predicting personality traits based on online posts has emerged as an important task in many fields such as social network analysis. One of the challenges of this task is assembling information from various posts into an overall profile for each user. While many previous solutions simply concatenate the posts into a long text and then encode the text by sequential or hierarchical models, they introduce unwarranted orders for the posts, which may mislead the models. In this paper, we propose a dynamic deep graph convolutional network (D-DGCN) to overcome the above limitation. Specifically, we design a learn-to-connect approach that adopts a dynamic multi-hop structure instead of a deterministic structure, and combine it with the DGCN module to automatically learn the connections between posts. The modules of post encoder, learn-to-connect, and DGCN are jointly trained in an end-to-end manner. Experimental results on the Kaggle and Pandora datasets show the superior performance of D-DGCN to state-of-the-art baselines. Our code is available at https://github.com/djz233/D-DGCN.

Interpreting Deep Graph Convolutional Networks with Spectrum Perspective

Graph convolutional network (GCN) architecture is the basis of many neural networks and has been widely used in processing graph-structured data. When dealing with large and sparse data, deeper GCN models are often required. However, the models suffer from performance degradation as the number of layers increases. The mainstream attribution of the current research is over-smoothing, and there are also gradient vanishing, training difficulties, etc., so a consensus cannot be reached. In this paper, we theoretically analyze the degradation problem by adopting spectral graph theory to globally consider the propagation and transformation components of the GCN architecture, and conclude that the over-smoothing problem caused by the propagation matrices is not the key factor for performance degradation. Afterwards, in addition to using conventional experimental methods, we proposed an experimental analysis strategy under the guidance of random matrix theory to analyze the singular value distribution of the model weight matrix. We concluded that the key factor leading to the degradation of model performance is the transformation component. In the context of a lack of consensus on the problem of model performance degradation, the paper proposes a systematic analysis strategy, as well as theoretical and empirical evidence.

GraphPLBR: Protein-ligand Binding Residue Prediction with Deep Graph Convolution Network.

The intermolecular interactions between proteins and ligands occur through site-specific amino acid residues in the proteins, and the identification of these key residues plays a critical role in both interpreting protein function and facilitating drug design based on virtual screening. In general, the information about the ligands-binding residues on proteins is unknown, and the detection of the binding residues by the biological wet experiments is time consuming. Therefore, many computational methods have been developed to identify the protein-ligand binding residues in recent years. We propose GraphPLBR, a framework based on Graph Convolutional Neural (GCN) networks, to predict protein-ligand binding residues (PLBR). The proteins are represented as a graph with residues as nodes through 3D protein structure data, such that the PLBR prediction task is transformed into a graph node classification task. A deep graph convolutional network is applied to extract information from higher-order neighbors, and initial residue connection with identity mapping is applied to cope with the over-smoothing problem caused by increasing the number of graph convolutional layers. To the best of our knowledge, this is a more unique and innovative perspective that utilizes the idea of graph node classification for protein-ligand binding residues prediction. By comparing with some state-of-the-art methods, our method performs better on several metrics.