Deep Graph Research Articles

Pipe Failure Prediction in the Water Distribution System Using a Deep Graph Convolutional Network and Temporal Failure Series

Pipe Failure Prediction in the Water Distribution System Using a Deep Graph Convolutional Network and Temporal Failure Series

Dynamic attention aggregated missing spatial–temporal data imputation for traffic speed prediction

Traffic speed forecasting is indispensable in Intelligent Traffic Systems (ITS). The missing traffic speed due to sensors’ failure may hamper the accurate prediction of traffic speed. Different kinds of imputation techniques were employed to replace the missing values considering the spatial and temporal information. However, previous techniques did not consider the dynamic contribution of the neighboring nodes and failed to capture the complex properties of traffic speed data precisely. The architecture of the previous imputation models was infeasible in real-time applications in terms of scalability. Therefore, a novel machine learning-based imputation technique is proposed to generate the missing traffic speed using the spatial–temporal information and dynamic contribution of neighboring nodes. A new dataset considering the characteristics of the traffic speed data is generated from the existing dataset by exploiting the non-missing traffic speed of neighboring nodes. The newly formed dataset helps in the estimation of the dynamic contribution of the neighboring nodes using the linear regression algorithm. The non-missing speeds and estimated dynamic contribution of neighboring nodes assist in calculating the missing speeds of a node. Furthermore, a novel deep learning-based prediction model is introduced to forecast traffic speed accurately. The prediction model contains graph structure learning, a deep graph neural network with skip connections, GRU, multi-head soft attention, and a fully connected neural network. Graph structure learning is proposed based on the distribution of the congestion to represent the graph in an efficient way. The deep graph neural network with skip connections and GRU help in capturing topological and temporal information, respectively. The multi-head soft attention is designed to focus on every relevant temporal information at each time step for capturing global traffic information. Finally, the extracted spatial–temporal features are fed into the fully connected neural network to predict the traffic speed. The proposed imputation and prediction models are evaluated on two real-time datasets, METR-LA and PeMSD7 datasets, under different missing rates of different missing patterns. The proposed framework generates possible spatial–temporal information for efficient traffic congestion management and evicts erroneous issues in real-time.

EQGraphNet: Advancing single-station earthquake magnitude estimation via deep graph networks with residual connections

Magnitude estimation is a critical task in seismology, and conventional methods usually require dense seismic station arrays to provide data with sufficient spatiotemporal distribution. In this context, we propose the Earthquake Graph Network (EQGraphNet) to enhance the performance of single-station magnitude estimation. The backbone of the proposed model consists of eleven convolutional neural network layers and ten RCGL modules, where a RCGL combines a Residual Connection and a Graph convolutional Layer capable of mitigating the over-smoothing problem and simultaneously extracting temporal features of seismic signals. Our work uses the STanford EArthquake Dataset for model training and performance testing. Compared with three existing deep learning models, EQGraphNet demonstrates improved accuracy for both local magnitude and duration magnitude scales. To evaluate the robustness, we add natural background noise to the model input and find that EQGraphNet achieves the best results, particularly for signals with lower signal-to-noise ratios. Additionally, by replacing various network components and comparing their estimation performances, we illustrate the contribution of each part of EQGraphNet, validating the rationality of our approach. We also demonstrate the generalization capability of our model across different earthquakes occurring environments, achieving mean errors of ±0.1 units. Furthermore, by demonstrating the effectiveness of deeper architectures, this work encourages further exploration of deeper GNN models for both multi-station and single-station magnitude estimation.

Auto-Correlation and Channel Attention Enhanced Deep Graph Convolution Networks for Gait Phase Prediction Based on Multi-IMU System

Gait phase prediction is important in controlling assistive robotic devices such as exoskeletons, where the control unit must differentiate between gait phases to provide the necessary assistance when the user is wearing the exoskeleton. To achieve the objective of precisely identifying the gait phase of users for the accurate control of the exoskeleton, this study proposes Auto-Correlation and Channel Attention enhanced Deep Graph Convolutional Networks (ACCA-DGCN) for gait phase prediction, and a gait phase prediction model based on multiple inertial measurement units (IMUs) and skeleton graph was established, in order to fully utilize the dependency among joints, and enhance accuracy and reliability of gait phase prediction. First, a human lower limb gait data acquisition equipment was developed, and the gait data of human walking were collected. The skeleton graph of the human lower limb was constructed through the natural connection relationship of joints in the human skeleton. After that, the ACCA-DGCN-based gait phase prediction model was constructed by using the gait data of human walking. Auto-Correlation (AC) and Efficient Channel Attention (ECA) were introduced to effectively capture periodic features of gait data and focus on the channels with high contributions to gait phase prediction. Finally, the effect of the window size on the performance of the ACCA-DGCN model was explored, and the proposed algorithm was compared with the other five deep learning algorithms: CNN, RNN, TCN, LSTM, and DGCN. The experimental results show that the average accuracy of gait phase prediction model based on ACCA-DGCN reaches up to 92.26% and 97.21% in user-independent and user-dependent experiments, respectively, which is superior to the other five algorithms. This study provides a new method for gait phase prediction, which is useful for improving the control of exoskeleton robots.

SGG-DGCN: Wind Turbine Anomaly Identification by Using Deep Graph Convolutional Networks with Similarity Graph Generation Strategy

In order to minimize wind turbine failures, fault diagnosis of wind turbines is becoming increasingly important, deep learning methods excel at multivariate monitoring and data modeling, but they are often limited to Euclidean space and struggle to capture the complex coupling between wind turbine sensors. To address this problem, we convert SCADA data into graph data, where sensors act as nodes and their topological connections act as edges, to represent these complex relationships more efficiently. Specifically, a wind turbine anomaly identification method based on deep graph convolutional neural network using similarity graph generation strategy (SGG-DGCN) is proposed. Firstly, a plurality of similarity graphs containing similarity information between nodes are generated by different distance metrics. Then, the generated similarity graphs are fused using the proposed similarity graph generation strategy. Finally, the fused similarity graphs are fed into the DGCN model for anomaly identification. To verify the effectiveness of the proposed SGG-DGCN model, we conducted a large number of experiments. The experimental results show that the proposed SGG-DGCN model has the highest accuracy compared with other models. In addition, the results of ablation experiment also demonstrate that the proposed SGG strategy can effectively improve the accuracy of WT anomaly identification. Conflict of Interest Statement The authors declare no conflicts of interest.

ProcGCN: detecting malicious process in memory based on DGCNN.

The combination of memory forensics and deep learning for malware detection has achieved certain progress, but most existing methods convert process dump to images for classification, which is still based on process byte feature classification. After the malware is loaded into memory, the original byte features will change. Compared with byte features, function call features can represent the behaviors of malware more robustly. Therefore, this article proposes the ProcGCN model, a deep learning model based on DGCNN (Deep Graph Convolutional Neural Network), to detect malicious processes in memory images. First, the process dump is extracted from the whole system memory image; then, the Function Call Graph (FCG) of the process is extracted, and feature vectors for the function node in the FCG are generated based on the word bag model; finally, the FCG is input to the ProcGCN model for classification and detection. Using a public dataset for experiments, the ProcGCN model achieved an accuracy of 98.44% and an F1 score of 0.9828. It shows a better result than the existing deep learning methods based on static features, and its detection speed is faster, which demonstrates the effectiveness of the method based on function call features and graph representation learning in memory forensics.

When Do Quantum Mechanical Descriptors Help Graph Neural Networks to Predict Chemical Properties?

Deep graph neural networks are extensively utilized to predict chemical reactivity and molecular properties. However, because of the complexity of chemical space, such models often have difficulty extrapolating beyond the chemistry contained in the training set. Augmenting the model with quantum mechanical (QM) descriptors is anticipated to improve its generalizability. However, obtaining QM descriptors often requires CPU-intensive computational chemistry calculations. To identify when QM descriptors help graph neural networks predict chemical properties, we conduct a systematic investigation of the impact of atom, bond, and molecular QM descriptors on the performance of directed message passing neural networks (D-MPNNs) for predicting 16 molecular properties. The analysis surveys computational and experimental targets, as well as classification and regression tasks, and varied data set sizes from several hundred to hundreds of thousands of data points. Our results indicate that QM descriptors are mostly beneficial for D-MPNN performance on small data sets, provided that the descriptors correlate well with the targets and can be readily computed with high accuracy. Otherwise, using QM descriptors can add cost without benefit or even introduce unwanted noise that can degrade model performance. Strategic integration of QM descriptors with D-MPNN unlocks potential for physics-informed, data-efficient modeling with some interpretability that can streamline de novo drug and material designs. To facilitate the use of QM descriptors in machine learning workflows for chemistry, we provide a set of guidelines regarding when and how to best leverage QM descriptors, a high-throughput workflow to compute them, and an enhancement to Chemprop, a widely adopted open-source D-MPNN implementation for chemical property prediction.

An attribution graph-based interpretable method for CNNs

Convolutional Neural Networks (CNNs) have demonstrated outstanding performance in various domains, such as face recognition, object detection, and image segmentation. However, the lack of transparency and limited interpretability inherent in CNNs pose challenges in fields such as medical diagnosis, autonomous driving, finance, and military applications. Several studies have explored the interpretability of CNNs and proposed various post-hoc interpretable methods. The majority of these methods are feature-based, focusing on the influence of input variables on outputs. Few methods undertake the analysis of parameters in CNNs and their overall structure. To explore the structure of CNNs and intuitively comprehend the role of their internal parameters, we propose an Attribution Graph-based Interpretable method for CNNs (AGIC) which models the overall structure of CNNs as graphs and provides interpretability from global and local perspectives. The runtime parameters of CNNs and feature maps of each image sample are applied to construct attribution graphs (At-GCs), where the convolutional kernels are represented as nodes and the SHAP values between kernel outputs are assigned as edges. These At-GCs are then employed to pretrain a newly designed heterogeneous graph encoder based on Deep Graph Infomax (DGI). To comprehensively delve into the overall structure of CNNs, the pretrained encoder is used for two types of interpretable tasks: (1) a classifier is attached to the pretrained encoder for the classification of At-GCs, revealing the dependency of At-GC’s topological characteristics on the image sample categories, and (2) a scoring aggregation (SA) network is constructed to assess the importance of each node in At-GCs, thus reflecting the relative importance of kernels in CNNs. The experimental results indicate that the topological characteristics of At-GC exhibit a dependency on the sample category used in its construction, which reveals that kernels in CNNs show distinct combined activation patterns for processing different image categories, meanwhile, the kernels that receive high scores from SA network are crucial for feature extraction, whereas low-scoring kernels can be pruned without affecting model performance, thereby enhancing the interpretability of CNNs.

Proactive Bandwidth Allocation for V2X Networks With Multi-Attentional Deep Graph Learning

Proactive Bandwidth Allocation for V2X Networks With Multi-Attentional Deep Graph Learning

Introducing new node prediction in graph mining: Predicting all links from isolated nodes with Graph Neural Networks

This paper introduces new node prediction, a fresh problem in the field of graph mining and social network analysis. This task can be categorized as zero-shot out-of-graph all-links prediction, and aims to predict all links coming from or going to a new, isolated, and unobserved node that was previously disconnected from the graph. In comparison with classic approaches to link prediction (including few-shot out-of-graph link prediction), this problem bears two key differences: (1) the new node has no existing links from which to extract patterns for new predictions; and (2) the goal is to predict not just one, but all the links of this new node, or, at least, a significant part of them. Experiments used an architecture based on Deep Graph Neural Networks, and were carried out on two datasets: a bibliographic citation graph and a drug–drug interaction graph. Results demonstrate that this challenging problem can be solved satisfactorily by using state-of-the-art Deep Learning techniques.

Task offloading and resource allocation in NOMA-VEC: A multi-agent deep graph reinforcement learning algorithm

Task offloading and resource allocation in NOMA-VEC: A multi-agent deep graph reinforcement learning algorithm

Prediction of Frequency-Dependent Optical Spectrum for Solid Materials: A Multioutput and Multifidelity Machine Learning Approach.

The frequency-dependent optical spectrum is pivotal for a broad range of applications from material characterization to optoelectronics and energy harvesting. Data-driven surrogate models, trained on density functional theory (DFT) data, have effectively alleviated the scalability limitations of DFT while preserving its chemical accuracy, expediting material discovery. However, prevailing machine learning (ML) efforts often focus on scalar properties such as the band gap, overlooking the complexities of optical spectra. In this work, we employ deep graph neural networks (GNNs) to predict the frequency-dependent complex-valued dielectric function across the infrared, visible, and ultraviolet spectra directly from the crystal structures. We explore multiple architectures for the spectral multioutput representation of the dielectric function and utilize various multifidelity learning strategies, such as transfer learning and fidelity embedding, to address the challenges associated with the scarcity of high-fidelity DFT data. Additionally, we model key solar cell absorption efficiency metrics, demonstrating that learning these parameters is enhanced when integrated through a learning bias within the learning of the frequency-dependent absorption coefficient. This study demonstrates that leveraging multioutput and multifidelity ML techniques enables accurate predictions of optical spectra from crystal structures, providing a versatile tool for rapidly screening materials for optoelectronics, optical sensing, and solar energy applications across an extensive frequency spectrum.

Improved Dual Correlation Reduction Network With Affinity Recovery.

Deep graph clustering, which aims to reveal the underlying graph structure and divide the nodes into different clusters without human annotations, is a fundamental yet challenging task. However, we observe that the existing methods suffer from the representation collapse problem and tend to encode samples with different classes into the same latent embedding. Consequently, the discriminative capability of nodes is limited, resulting in suboptimal clustering performance. To address this problem, we propose a novel deep graph clustering algorithm termed improved dual correlation reduction network (IDCRN) through improving the discriminative capability of samples. Specifically, by approximating the cross-view feature correlation matrix to an identity matrix, we reduce the redundancy between different dimensions of features, thus improving the discriminative capability of the latent space explicitly. Meanwhile, the cross-view sample correlation matrix is forced to approximate the designed clustering-refined adjacency matrix to guide the learned latent representation to recover the affinity matrix even across views, thus enhancing the discriminative capability of features implicitly. Moreover, we avoid the collapsed representation caused by the oversmoothing issue in graph convolutional networks (GCNs) through an introduced propagation regularization term, enabling IDCRN to capture the long-range information with the shallow network structure. Extensive experimental results on six benchmarks have demonstrated the effectiveness and efficiency of IDCRN compared with the existing state-of-the-art deep graph clustering algorithms. The code of IDCRN is released at https://github.com/yueliu1999/IDCRN. Besides, we share a collection of deep graph clustering, including papers, codes, and datasets at https://github.com/yueliu1999/Awesome-Deep-Graph-Clustering.

Structured Deep Graph Clustering Network Based on Consistency Constraint

Graph clustering is an essential task in data analysis. Recently, there has been a notable trend in the application of deep learning in graph clustering. However, it is worth noting that existing deep graph clustering methods primarily focus on the topological information of nodes while overlooking the use of attribute information in the learned feature representations. In addition, they do not address the consistency of the space distribution in attribute and structure clustering results. To tackle these critical issues, a novel structured deep graph clustering network with consistency constraint (CC-DGC) is proposed. The network first constructs an autoencoder to learn and transmit hierarchical attribute information to the graph autoencoder (GAE). Subsequently, the GAE integrates the received hierarchical attribute information with extracted topological information to generate enhanced clustering representations. Moreover, this paper designs a consistency constraint module to promote consistency between the autoencoder and the GAE by optimizing the cluster space distributions they produce. Finally, the feature extraction and clustering classification processes are synchronized and optimized in a self-supervised manner within a unified framework. Extensive experiments illustrate that the proposed CC-DGC demonstrates superiority over the state-of-the-art deep graph clustering methods on five benchmark datasets.

Performance Assessment of Universal Machine Learning Interatomic Potentials: Challenges and Directions for Materials' Surfaces.

Machine learning interatomic potentials (MLIPs) are one of the main techniques in the materials science toolbox, able to bridge ab initio accuracy with the computational efficiency of classical force fields. This allows simulations ranging from atoms, molecules, and biosystems, to solid and bulk materials, surfaces, nanomaterials, and their interfaces and complex interactions. A recent class of advanced MLIPs, which use equivariant representations and deep graph neural networks, is known as universal models. These models are proposed as foundation models suitable for any system, covering most elements from the periodic table. Current universal MLIPs (UIPs) have been trained with the largest consistent data set available nowadays. However, these are composed mostly of bulk materials' DFT calculations. In this article, we assess the universality of all openly available UIPs, namely MACE, CHGNet, and M3GNet, in a representative task of generalization: calculation of surface energies. We find that the out-of-the-box foundation models have significant shortcomings in this task, with errors correlated to the total energy of surface simulations, having an out-of-domain distance from the training data set. Our results show that while UIPs are an efficient starting point for fine-tuning specialized models, we envision the potential of increasing the coverage of the materials space toward universal training data sets for MLIPs.

Large-scale prediction of collision cross-section with very deep graph convolutional network for small molecule identification

Ion mobility spectrometry (IMS) is a promising analytical technique for mass spectrometry (MS)-based compound identification by providing collision cross-section (CCS) value as an additional dimension with structural information. Here, GraphCCS was proposed to accurately predict the CCS value and expand the coverage of CCS libraries. A new adduct encoding method was proposed to encode SMILES strings and adduct types of compounds into adduct graphs. GraphCCS extended its predictive capability to ten different adduct types. A very deep graph convolutional network with up to 40 GCN layers was built to predict CCS values from adduct graphs. A curated dataset with 12,775 experimental CCS values was used to train, validate, and test the GraphCCS model. The resulting CCS predictions achieved a median relative error (MedRE) of 0.94 % and a coefficient of determination (R2) of 0.994 on the test set. Results on external test sets showed that GraphCCS outperformed AllCCS2, CCSbase, SigmaCCS, and DeepCCS. Based on the developed GraphCCS method, a large-scale in-silico database was built, including 2,394,468 CCS values. Those CCS values can be used to filter false positives complementary to retention times and tandem mass spectra. Finally, the effectiveness of GraphCCS in assisting compound identification was tested on a mouse adrenal gland lipid dataset with 1,960 lipids. The results demonstrated that the in-silico CCS values combined with MS spectra and retention times can efficiently filter the false positive candidates.

Learnable Graph Matching: A Practical Paradigm for Data Association.

Data association is at the core of many computer vision tasks, e.g., multiple object tracking, image matching, and point cloud registration. however, current data association solutions have some defects: they mostly ignore the intra-view context information; besides, they either train deep association models in an end-to-end way and hardly utilize the advantage of optimization-based assignment methods, or only use an off-the-shelf neural network to extract features. In this paper, we propose a general learnable graph matching method to address these issues. Especially, we model the intra-view relationships as an undirected graph. Then data association turns into a general graph matching problem between graphs. Furthermore, to make optimization end-to-end differentiable, we relax the original graph matching problem into continuous quadratic programming and then incorporate training into a deep graph neural network with KKT conditions and implicit function theorem. In MOT task, our method achieves state-of-the-art performance on several MOT datasets. For image matching, our method outperforms state-of-the-art methods on a popular indoor dataset, ScanNet. For point cloud registration, we also achieve competitive results.

Node-Smoothness-Based Adaptive Initial Residual Deep Graph Convolutional Network

Node-Smoothness-Based Adaptive Initial Residual Deep Graph Convolutional Network

Unraveling Spatial Domain Characterization in Spatially Resolved Transcriptomics with Robust Graph Contrastive Clustering.

Spatial transcriptomics can quantify gene expression and its spatial distribution in tissues, thus revealing molecular mechanisms of cellular interactions underlying tissue heterogeneity, tissue regeneration, and spatially localized disease mechanisms. However, existing spatial clustering methods often fail to exploit the full potential of spatial information, resulting in inaccurate identification of spatial domains. In this paper, we develop a deep graph contrastive clustering framework, stDGCC, that accurately uncovers underlying spatial domains via explicitly modeling spatial information and gene expression profiles from spatial transcriptomics data. The stDGCC framework proposes a spatially informed graph node embedding model to preserve the topological information of spots and to learn the informative and discriminative characterization of spatial transcriptomics data through self-supervised contrastive learning. By simultaneously optimizing the contrastive learning loss, reconstruction loss, and Kullback-Leibler (KL) divergence loss, stDGCC achieves joint optimization of feature learning and topology structure preservation in an end-to-end manner. We validate the effectiveness of stDGCC on various spatial transcriptomics datasets acquired from different platforms, each with varying spatial resolutions. Our extensive experiments demonstrate the superiority of stDGCC over various state-of-the-art clustering methods in accurately identifying cellular-level biological structures. Code and data are available from https://github.com/TimE9527/stDGCC and https://figshare.com/projects/stDGCC/186525. Supplementary data are available at Bioinformatics online.

A Unified Framework for Neurological Disease Detection and Gait Classification Using Deep Graph Learning

A Unified Framework for Neurological Disease Detection and Gait Classification Using Deep Graph Learning