Hybrid Graph Research Articles

Learning and integration of adaptive hybrid graph structures for multivariate time series forecasting

Recent status-of-the-art methods for multivariate time series forecasting can be categorized into graph-based approach and global-local approach. The former approach uses graphs to represent the dependencies among variables and apply graph neural networks to the forecasting problem. The latter approach decomposes the matrix of multivariate time series into global components and local components to capture the shared information across variables. However, both approaches cannot capture the propagation delay of the dependencies among individual variables of a multivariate time series, for example, the congestion at intersection A has delayed effects on the neighboring intersection B. In addition, graph-based forecasting methods cannot capture the shared global tendency across the variables of a multivariate time series; and global-local forecasting methods cannot reflect the nonlinear inter-dependencies among variables of a multivariate time series. In this paper, we propose to combine the advantages of both approaches by integrating Adaptive Global-Local Graph Structure Learning with Gated Recurrent Units (AGLG-GRU). We learn a global graph to represent the shared information across variables. And we learn dynamic local graphs to capture the local randomness and nonlinear dependencies among variables. We apply diffusion convolution and graph convolution operations to global and dynamic local graphs to integrate the information of graphs and update gated recurrent unit for multivariate time series forecasting. The experimental results on seven representative real-world datasets demonstrate that our approach outperforms various existing methods.

Hybrid Graph Convolutional Network With Online Masked Autoencoder for Robust Multimodal Cancer Survival Prediction.

Cancer survival prediction requires exploiting related multimodal information (e.g., pathological, clinical and genomic features, etc.) and it is even more challenging in clinical practices due to the incompleteness of patient's multimodal data. Furthermore, existing methods lack sufficient intra- and inter-modal interactions, and suffer from significant performance degradation caused by missing modalities. This manuscript proposes a novel hybrid graph convolutional network, entitled HGCN, which is equipped with an online masked autoencoder paradigm for robust multimodal cancer survival prediction. Particularly, we pioneer modeling the patient's multimodal data into flexible and interpretable multimodal graphs with modality-specific preprocessing. HGCN integrates the advantages of graph convolutional networks (GCNs) and a hypergraph convolutional network (HCN) through node message passing and a hyperedge mixing mechanism to facilitate intra-modal and inter-modal interactions between multimodal graphs. With HGCN, the potential for multimodal data to create more reliable predictions of patient's survival risk is dramatically increased compared to prior methods. Most importantly, to compensate for missing patient modalities in clinical scenarios, we incorporated an online masked autoencoder paradigm into HGCN, which can effectively capture intrinsic dependence between modalities and seamlessly generate missing hyperedges for model inference. Extensive experiments and analysis on six cancer cohorts from TCGA show that our method significantly outperforms the state-of-the-arts in both complete and missing modal settings. Our codes are made available at https://github.com/lin-lcx/HGCN.

Hybrid Graph Models for Traffic Prediction

Obtaining accurate road conditions is crucial for traffic management, dynamic route planning, and intelligent guidance services. The complex spatial correlation and nonlinear temporal dependence pose great challenges to obtaining accurate road conditions. Existing graph-based methods use a static adjacency matrix or a dynamic adjacency matrix to aggregate spatial information between nodes, which cannot fully represent the topological information. In this paper, we propose a Hybrid Graph Model (HGM) for accurate traffic prediction. The HGM constructs a static graph and a dynamic graph to represent the topological information of the traffic network, which is beneficial for mining potential and obvious spatial correlations. The proposed method combines a graph neural network, convolutional neural network, and attention mechanism to jointly extract complex spatial–temporal features. The HGM consists of two different sub-modules, called spatial–temporal attention module and dynamic graph convolutional network, to fuse complex spatial–temporal information. Furthermore, the proposed method designs a novel gated function to adaptively fuse the results from spatial–temporal attention and dynamic graph convolutional network to improve prediction performance. Extensive experiments on two real datasets show that the HGM outperforms comparable state-of-the-art methods.

An Efficient Hybrid Graph Network Model for Traveling Salesman Problem with Drone

An Efficient Hybrid Graph Network Model for Traveling Salesman Problem with Drone

Hybrid graph transformer networks for multivariate time series anomaly detection

Hybrid graph transformer networks for multivariate time series anomaly detection

Conditional Diffusion Based on Discrete Graph Structures for Molecular Graph Generation

Learning the underlying distribution of molecular graphs and generating high-fidelity samples is a fundamental research problem in drug discovery and material science. However, accurately modeling distribution and rapidly generating novel molecular graphs remain crucial and challenging goals. To accomplish these goals, we propose a novel Conditional Diffusion model based on discrete Graph Structures (CDGS) for molecular graph generation. Specifically, we construct a forward graph diffusion process on both graph structures and inherent features through stochastic differential equations (SDE) and derive discrete graph structures as the condition for reverse generative processes. We present a specialized hybrid graph noise prediction model that extracts the global context and the local node-edge dependency from intermediate graph states. We further utilize ordinary differential equation (ODE) solvers for efficient graph sampling, based on the semi-linear structure of the probability flow ODE. We also combine the solvers with gradient guidance from the molecule property predictor for similarity-constrained molecule optimization. Experiments on diverse datasets validate the effectiveness of our framework. Particularly, the proposed method still generates high-quality molecular graphs in a limited number of steps.

Joint Latent Space and Label Inference Estimation with Adaptive Fused Data and Label Graphs

Recently, structured computing has become an interesting topic in the world of artificial intelligence, especially in the field of machine learning, as most researchers focus on the development of graph-based semi-supervised learning models. In this article, we present a new framework for graph-based semi-supervised learning. We present a powerful method for simultaneous label inference and linear transform estimation. The targeted linear transformation is used to obtain a discriminant subspace. To improve semi-supervised learning, our framework focuses on exploiting the data structure and soft labels of the available unlabeled samples. In the iterative optimization scheme used, the prior estimation of the label increases the supervision information indirectly through an introduced informative matrix called the label graph, thus avoiding the use of hard confidence-based decisions as used in self-supervised methods. In addition, the estimation of labels and projected data is made more robust by using smoothing concepts based on hybrid graphs. For each type of smoothing, the hybrid graph is an adaptive fusion of the two graphs encoding the similarity of the data and the similarity of the labels. The proposed method leads to an improved discriminative linear transformation. Several experimental results on real image datasets confirm the effectiveness of the proposed method. They also show superior performance compared to semi-supervised methods that use integration and label inference simultaneously.

In silico prediction of drug-induced liver injury with a complementary integration strategy based on hybrid representation.

Drug-induced liver injury (DILI) is one of the major causes of drug withdrawals, acute liver injury and blackbox warnings. Clinical diagnosis of DILI is a huge challenge due to the complex pathogenesis and lack of specific biomarkers. In recent years, machine learning methods have been used for DILI risk assessment, but the model generalization does not perform satisfactorily. In this study, we constructed a large DILI data set and proposed an integration strategy based on hybrid representations for DILI prediction (HR-DILI). Benefited from feature integration, the hybrid graph neural network models outperformed single representation-based models, among which hybrid-GraphSAGE showed balanced performance in cross-validation with AUC (area under the curve) as 0.804 ± 0.019. In the external validation set, HR-DILI improved the AUC by 6.4%-35.9% compared to the base model with a single representation. Compared with published DILI prediction models, HR-DILI had better and balanced performance. The performance of local models for natural products and synthetic compounds were also explored. Furthermore, eight key descriptors and six structural alerts associated with DILI were analyzed to increase the interpretability of the models. The improved performance of HR-DILI indicated that it would provide reliable guidance for DILI risk assessment.

Hi-MGT: A hybrid molecule graph transformer for toxicity identification

Conventional toxicity testing methods that rely on animal experimentation are resource-intensive, time-consuming, and ethically controversial. Therefore, the development of alternative non-animal testing approaches is crucial. This study proposes a novel hybrid graph transformer architecture, termed Hi-MGT, for the toxicity identification. An innovative aggregation strategy, referred to as GNN-GT combination, enables Hi-MGT to simultaneously and comprehensively aggregate local and global structural information of molecules, thus elucidating more informative toxicity information hidden in molecule graphs. The results show that the state-of-the-art model outperforms current baseline CML and DL models on a diverse range of toxicity endpoints and is even comparable to large-scale pretrained GNNs with geometry enhancement. Additionally, the impact of hyperparameters on model performance is investigated, and a systematic ablation study is conducted to demonstrate the effectiveness of the GNN-GT combination. Moreover, this study provides valuable insights into the learning process on molecules and proposes a novel similarity-based method for toxic site detection, which could potentially facilitate toxicity identification and analysis. Overall, the Hi-MGT model represents a significant advancement in the development of alternative non-animal testing approaches for toxicity identification, with promising implications for enhancing human safety in the use of chemical compounds.

NHGNN-DTA: a node-adaptive hybrid graph neural network for interpretable drug-target binding affinity prediction.

Large-scale prediction of drug-target affinity (DTA) plays an important role in drug discovery. In recent years, machine learning algorithms have made great progress in DTA prediction by utilizing sequence or structural information of both drugs and proteins. However, sequence-based algorithms ignore the structural information of molecules and proteins, while graph-based algorithms are insufficient in feature extraction and information interaction. In this article, we propose NHGNN-DTA, a node-adaptive hybrid neural network for interpretable DTA prediction. It can adaptively acquire feature representations of drugs and proteins and allow information to interact at the graph level, effectively combining the advantages of both sequence-based and graph-based approaches. Experimental results have shown that NHGNN-DTA achieved new state-of-the-art performance. It achieved the mean squared error (MSE) of 0.196 on the Davis dataset (below 0.2 for the first time) and 0.124 on the KIBA dataset (3% improvement). Meanwhile, in the case of cold start scenario, NHGNN-DTA proved to be more robust and more effective with unseen inputs than baseline methods. Furthermore, the multi-head self-attention mechanism endows the model with interpretability, providing new exploratory insights for drug discovery. The case study on Omicron variants of SARS-CoV-2 illustrates the efficient utilization of drug repurposing in COVID-19. The source code and data are available at https://github.com/hehh77/NHGNN-DTA.

72-hour real-time forecasting of ambient PM2.5 by hybrid graph deep neural network with aggregated neighborhood spatiotemporal information

The observation-based air pollution forecasting method has high computational efficiency over traditional numerical models, but a poor ability in long-term (after 6 h) forecasting due to a lack of detailed representation of atmospheric processes associated with the pollution transport. To address such limitation, here we propose a novel real-time air pollution forecasting model that applies a hybrid graph deep neural network (GNN_LSTM) to dynamically capture the spatiotemporal correlations among neighborhood monitoring sites to better represent the physical mechanism of pollutant transport across the space with the graph structure which is established with features (angle, wind speed, and wind direction) of neighborhood sites to quantify their interactions. Such design substantially improves the model performance in 72-hour PM2.5 forecasting over the whole Beijing-Tianjin-Hebei region (overall R2 increases from 0.6 to 0.79), particularly for polluted episodes (PM2.5 concentration > 55 µg/m3) with pronounced regional transport to be captured by GNN_LSTM model. The inclusion of the AOD feature further enhances the model performance in predicting PM2.5 over the sites where the AOD can inform additional aloft PM2.5 pollution features related to regional transport. The importance of neighborhood site (particularly for those in the upwind flow pathway of the target area) features for long-term PM2.5 forecast is demonstrated by the increased performance in predicting PM2.5 in the target city (Beijing) with the inclusion of additional 128 neighborhood sites. Moreover, the newly developed GNN_LSTM model also implies the “source”-receptor relationship, as impacts from distanced sites associated with regional transport grow along with the forecasting time (from 0% to 38% in 72 h) following the wind flow. Such results suggest the great potential of GNN_LSTM in long-term air quality forecasting and air pollution prevention.

MS_HGNN: a hybrid online fraud detection model to alleviate graph-based data imbalance

Online transaction fraud has become increasingly rampant due to the convenience of mobile payment. Fraud detection is critical to ensure the security of online transactions. With the development of graph neural network, researchers have applied it to the field of fraud detection. The existing fraud detection methods will solve the class imbalance by sampling, but they do not fully consider the various imbalances in the heterogeneous graph, and the data imbalance will directly affect the performance of the model. This work proposes a hybrid graph neural network model for online fraud detection to address this issue. The three types of imbalance in online transactions are feature imbalance, category imbalance, and relation imbalance, and they are all addressed in the proposed model. The entities with the feature most closely related to the fraudsters will be determined for the feature imbalance, and samples will be taken for further identification in the subsequent training phase. The hybrid model then uses under-sampling in combination with the long-distance sampling to find nodes with high similarity of features for the category imbalance. Finally, we propose a reward/punishment mechanism based on reinforcement learning for relation imbalance, which uses the threshold created by training as the sampling weight between relations. This paper conducts experiments on the public datasets Amazon and Yelp. The experimental results show that the model proposed is 5.61% higher than the best model in the comparison model on Amazon dataset, and 1.58% higher on Yelp dataset.

Hybrid Graph Neural Network Recommendation Based on Multi-Behavior Interaction and Time Sequence Awareness

In recent years, mining user multi-behavior information for prediction has become a hot topic in recommendation systems. Usually, researchers only use graph networks to capture the relationship between multiple types of user-interaction information and target items, while ignoring the order of interactions. This makes multi-behavior information underutilized. In response to the above problem, we propose a new hybrid graph network recommendation model called the User Multi-Behavior Graph Network (UMBGN). The model uses a joint learning mechanism to integrate user–item multi-behavior interaction sequences. We designed a user multi-behavior information-aware layer to focus on the long-term multi-behavior features of users and learn temporally ordered user–item interaction information through BiGRU units and AUGRU units. Furthermore, we also defined the propagation weights between the user–item interaction graph and the item–item relationship graph according to user behavior preferences to capture more valuable dependencies. Extensive experiments on three public datasets, namely MovieLens, Yelp2018, and Online Mall, show that our model outperforms the best baselines by 2.04%, 3.82%, and 3.23%.

Improving anatomical plausibility in medical image segmentation via hybrid graph neural networks: applications to chest x-ray analysis.

Anatomical segmentation is a fundamental task in medical image computing, generally tackled with fully convolutional neural networks which produce dense segmentation masks. These models are often trained with loss functions such as cross-entropy or Dice, which assume pixels to be independent of each other, thus ignoring topological errors and anatomical inconsistencies. We address this limitation by moving from pixel-level to graph representations, which allow to naturally incorporate anatomical constraints by construction. To this end, we introduce HybridGNet, an encoder-decoder neural architecture that leverages standard convolutions for image feature encoding and graph convolutional neural networks (GCNNs) to decode plausible representations of anatomical structures. We also propose a novel image-to-graph skip connection layer which allows localized features to flow from standard convolutional blocks to GCNN blocks, and show that it improves segmentation accuracy. The proposed architecture is extensively evaluated in a variety of domain shift and image occlusion scenarios, and audited considering different types of demographic domain shift. Our comprehensive experimental setup compares HybridGNet with other landmark and pixel-based models for anatomical segmentation in chest x-ray images, and shows that it produces anatomically plausible results in challenging scenarios where other models tend to fail.

Hybrid Graph Partitioning with OLB Approach in Distributed Transactions

Online Transaction Processing (OLTP) gets support from data partitioning to achieve better performance and scalability. The primary objective of database and application developers is to provide scalable and reliable database systems. This research presents a novel method for data partitioning and load balancing for scalable transactions. Data is efficiently partitioned using the hybrid graph partitioning method. Optimized load balancing (OLB) approach is applied to calculate the weight factor, average workload, and partition efficiency. The presented approach is appropriate for various online data transaction applications. The quality of the proposed approach is examined using OLTP database benchmark. The performance of the proposed methodology significantly outperformed with respect to metrics like throughput, response time, and CPU utilization.

Towards Accurate Microstructure Estimation via 3D Hybrid Graph Transformer.

Deep learning has drawn increasing attention in microstructure estimation with undersampled diffusion MRI (dMRI) data. A representative method is the hybrid graph transformer (HGT), which achieves promising performance by integrating -space graph learning and -space transformer learning into a unified framework. However, this method overlooks the 3D spatial information as it relies on training with 2D slices. To address this limitation, we propose 3D hybrid graph transformer (3D-HGT), an advanced microstructure estimation model capable of making full use of 3D spatial information and angular information. To tackle the large computation burden associated with 3D -space learning, we propose an efficient -space learning model based on simplified graph neural networks. Furthermore, we propose a 3D -space learning module based on the transformer. Extensive experiments on data from the human connectome project show that our 3D-HGT outperforms state-of-the-art methods, including HGT, in both quantitative and qualitative evaluations.

Semi-supervised graph learning framework for apicomplexan parasite classification

Apicomplexan parasites cause diseases including malaria, toxoplasmosis, and babesiosis, affecting large parts of the world and hampering economic development to a considerable severity. Early recognition of pathogenic parasites can effectively constrain the spread of these infectious diseases. Recently, deep-learning-based computer-aided parasite recognition systems provide many practical tools for automated apicomplexan parasite recognition. However, existing research works are limited by the complicated and expensive manual annotating workload in supervised deep learning methods, which require many experts’ manually annotated samples. Moreover, they also ignore the significance of correlations among different parasite cells, which retain helpful information for identification. Therefore, this paper is the first to employ a semi-supervised learning strategy to reduce the manual labeling workload and introduce a hybrid graph learning algorithm into parasite recognition. Our approach has a superior parasite recognition capability with semi-supervised techniques and the graph-based model. Specifically, this research proposes a Semi-Supervised Graph Learning (SSGL) framework, composing CNN (Convolutional Neural Networks) feature embedding, learnable graph building, and graph feature learning components. To verify the performance of the SSGL model, the paper applies multiple evaluation indicators to appraise our model, and the approach can achieve high accuracy of 91.75%, AUC of 91.83%, the sensitivity of 91.75% and specificity of 97.25% with only a small amount of labeled data (20%). In conclusion, SSGL model provides high parasite recognition capability with minimal labels, greatly reduces the workload of relevant experts, and promotes the popularization of computer-aided parasite diagnosis tools.

Dynamic traffic correlations based spatio-temporal graph convolutional network for urban traffic prediction

Accurate urban traffic prediction is a critical issue in Intelligent Transportation Systems (ITS). It is challenging since urban traffic usually indicates high dynamic spatio-temporal correlations, leading to uncertainty and complexity of traffic status. Since the transportation network is a graph structure practically, existing works have applied Graph Convolutional Network (GCN) on urban traffic prediction with a pre-defined adjacency matrix based on node distance or connectivity. However, in many urban traffic scenarios, spatio-temporal dependencies among traffic data usually change over time, so using a fixed adjacency matrix cannot describe the dynamic dependencies. To track the dynamic spatio-temporal dependencies among traffic data, we propose a novel deep learning framework, Dynamic Traffic Correlation-based Spatio-Temporal Graph Convolutional network (DTC-STGCN), to forecast traffic flow and speed accurately. DTC-STGCN extracts a dynamic adjacency matrix from different traffic characters to describe dynamic spatio-temporal correlations. Moreover, an attention and dynamic adjacency matrix-based GCNs framework is proposed to capture urban traffic dynamic spatial features, while a long-short-term memory network (LSTM) is used to capture urban traffic temporal features, respectively. Finally, we feed the spatio-temporal features generated by GCN and LSTM, with real road segments into a hybrid graph convolution framework to simultaneously model the dynamic spatial and temporal dependencies for traffic predictions. The experiments on two real-world datasets demonstrate that the proposed DTC-STGCN model consistently outperforms the state-of-the-art traffic prediction baselines on MAE and RMSE over 10%, and achieve a stable performance for two specific tasks (long-term traffic prediction and peak time prediction). And ablation study validates the effectiveness of dynamic adjacency matrix, attention mechanism, respectively.

Learn from structural scope: Improving aspect-level sentiment analysis with hybrid graph convolutional networks

Aspect-level sentiment analysis aims to determine the sentiment polarity towards a specific target in a sentence. The main challenge of this task is to effectively model the relation between targets and sentiments so as to filter out noisy opinion words from irrelevant targets. Most recent efforts capture relations through target-sentiment pairs or opinion spans from a word-level or phrase-level perspective. Based on the observation that targets and sentiments essentially establish relations following the grammatical hierarchy of phrase-clause-sentence structure, it is hopeful to exploit comprehensive syntactic information for better guiding the learning process. Therefore, we introduce the concept of Scope, which outlines a structural text region related to a specific target. To jointly learn structural Scope and predict the sentiment polarity, we propose a hybrid graph convolutional network (HGCN) to synthesize information from constituency tree and dependency tree, exploring the potential of linking two syntax parsing methods to enrich the representation. Experimental results on five public datasets illustrate that our HGCN model outperforms current state-of-the-art baselines. More specifically, the average accuracy/ F1 score improvements of our HGCN compared to baseline models on Restaurant 14, 15 and 16 are 2.46%/5.36%, 2.25%/5.70% and 1.73%/5.50%, while the performance improvements are 3.32%/4.30% and 2.50%/3.08% on the Laptop and Twitter datasets, respectively. Furthermore, when cascaded to five models, our method has significantly improved their performances by simplifying the sentence from multiple targets to a single one.

Taming transitive redundancy for context-free language reachability

Given an edge-labeled graph, context-free language reachability (CFL-reachability) computes reachable node pairs by deriving new edges and adding them to the graph. The redundancy that limits the scalability of CFL-reachability manifests as redundant derivations, i.e., identical edges can be derived multiple times due to the many paths between two reachable nodes. We observe that most redundancy arises from the derivations involving transitive relations of reachable node pairs. Unfortunately, existing techniques for reducing redundancy in transitive-closure-based problems are either ineffective or inapplicable to identifying and eliminating redundant derivations during on-the-fly CFL-reachability solving. This paper proposes a scalable yet precision-preserving approach to all-pairs CFL-reachability analysis by taming its transitive redundancy. Our key insight is that transitive relations are intrinsically ordered, and utilizing the order for edge derivation can avoid most redundancy. To address the challenges in determining the derivation order from the dynamically changed graph during CFL-reachability solving, we introduce a hybrid graph representation by combining spanning trees and adjacency lists, together with a dynamic construction algorithm. Based on this representation, we propose a fast and effective partially ordered algorithm POCR to boost the performance of CFL-reachability analysis by reducing its transitive redundancy during on-the-fly solving. Our experiments on context-sensitive value-flow analysis and field-sensitive alias analysis for C/C++ demonstrate the promising performance of POCR. On average, POCR eliminates 98.50% and 97.26% redundant derivations respectively for the value-flow and alias analysis, achieving speedups of 21.48× and 19.57× over the standard CFL-reachability algorithm. We also compare POCR with two recent open-source tools, Graspan (a CFL-reachability solver) and Soufflé (a Datalog engine). The results demonstrate that POCR is over 3.67× faster than Graspan and Soufflé on average for both value-flow analysis and alias analysis.