Heterogeneous Graph Neural Network Research Articles

HDM-GNN: A Heterogeneous Dynamic Multi-view Graph Neural Network for Crime Prediction

HDM-GNN: A Heterogeneous Dynamic Multi-view Graph Neural Network for Crime Prediction

Flexible job shop scheduling via deep reinforcement learning with meta-path-based heterogeneous graph neural network

The flexible job shop scheduling problem (FJSP) is an important production scheduling problem in intelligent manufacturing. How to model the complex FJSP more accurately and improve the efficiency and generalization of scheduling policies is an urgent challenge to be solved. Therefore, a new end-to-end deep reinforcement learning (DRL) method combined with the meta-path-based heterogeneous graph neural network (MHGNN) is proposed to effectively solve FJSP. The task of solving FJSP is decomposed into two subtasks of operation selection and machine allocation. This dual-task FJSP is represented by introducing a heterogeneous graph and modeled as the dual Markov decision process (DMDP). A MHGNN is proposed to embed the global scheduling states of the dual-task FJSP. A heterogeneous graph neural network (GNN) framework is designed for the dual-task FJSP to efficiently encode the operation nodes and machine nodes, where the extracted embedded feature information is used to represent the operation selection policy and the machine allocation policy. Two policy networks are designed to efficiently predict the operation selection policy and the machine allocation policy, and a soft double-actors critic algorithm is proposed to train these two policy networks, where the trained policies can be used to efficiently solve FJSP instances of different scales. The experiments on three public benchmarks show that the proposed method outperforms the well-known heuristic scheduling rules and some advanced methods for solving FJSP. In particular, when solving large-scale FJSPs, the proposed method performs more prominently in terms of solution quality and solution time.

Relation-aware heterogeneous graph neural network for entity alignment

Entity alignment refers to finding equivalent entities from different knowledge graphs. Most of the existing entity alignment methods are studied based on homogeneous graphs. However, the knowledge graph is a heterogeneous graph containing many types of nodes, such as entities, relations, and attributes. Therefore, we propose introducing a heterogeneous graph neural network to model entities and relations simultaneously and propose an iterative fusion method to enhance the interaction between these two semantic nodes. Since not all datasets contain relation information, this paper does not directly introduce a feature representation of relations. The generalizability of the approach is improved by utilizing a relation-aware strategy to obtain information about the relation. Specifically, the information propagation of the head and tail entities in the triplet is utilized to obtain the feature representation of the relation. Experimental results show that the present method performs better on three cross-lingual datasets DBP15K and two large-scale datasets DWY100K.

Few-shot Learning for Heterogeneous Information Networks

Heterogeneous information networks (HINs) are a key resource in many domain-specific retrieval and recommendation scenarios and in conversational environments. Current approaches to mining graph data often rely on abundant supervised information. However, supervised signals for graph learning tend to be scarce for a new task and only a handful of labeled nodes may be available. Meta-learning mechanisms are able to harness prior knowledge that can be adapted to new tasks. In this article, we design meta-learning framework for heterogeneous information networks ( META-HIN ), for few-shot learning problems on HINs. To the best of our knowledge, we are among the first to design a unified framework to realize the few-shot learning of HINs and facilitate different downstream tasks across different domains of graphs. Unlike most previous models, which focus on a single task on a single graph, META-HIN is able to deal with different tasks (node classification, link prediction, and anomaly detection are used as examples) across multiple graphs. Subgraphs are sampled to build the support and query set. Before being processed by the meta-learning module, subgraphs are modeled via a structure module to capture structural features. Then, a heterogeneous Graph Neural Network module is used as the base model to express the features of subgraphs. We also design a Generative Adversarial Network-based contrastive learning module that is able to exploit unsupervised information of the subgraphs. In our experiments, we fuse several datasets from multiple domains to verify META-HIN ’s broad applicability in a multiple-graph scenario. META-HIN consistently and significantly outperforms state-of-the-art alternatives on every task and across all datasets that we consider.

Health insurance fraud detection based on multi-channel heterogeneous graph structure learning

Health insurance fraud is becoming more common and impacting the fairness and sustainability of the health insurance system. Traditional health insurance fraud detection primarily relies on recognizing established data patterns. However, with the ever-expanding and complex nature of health insurance data, it is difficult for these traditional methods to effectively capture evolving fraudulent activity and tactics and keep pace with the constant improvements and innovations of fraudsters. As a result, there is an urgent need for more accurate and flexible analytics to detect potential fraud. To address this, the Multi-channel Heterogeneous Graph Structured Learning-based health insurance fraud detection method (MHGSL) was proposed. MHGSL constructs a graph of health insurance data from various entities, such as patients, departments, and medicines, and employs graph structure learning to extract topological structure, features, and semantic information to construct multiple graphs that reflect the diversity and complexity of the data. We utilize deep learning methods such as heterogeneous graph neural networks and graph convolutional neural networks to combine multi-channel information transfer and feature fusion to detect anomalies in health insurance data. The results of extensive experiments on real health insurance data demonstrate that MHGSL achieves a high level of accuracy in detecting potential fraud, which is better than existing methods, and is able to quickly and accurately identify patients with fraudulent behaviors to avoid loss of health insurance funds. Experiments have shown that multi-channel heterogeneous graph structure learning in MHGSL can be very helpful for health insurance fraud detection. It provides a promising solution for detecting health insurance fraud and improving the fairness and sustainability of the health insurance system. Subsequent research on fraud detection methods can consider semantic information between patients and different types of entities.

Railway network delay evolution: A heterogeneous graph neural network approach

Accurate delay evolution prediction plays a pivotal role in train rescheduling decision-making for the railway network. Existing studies on delay prediction predominantly centered around predicting delays for each train in the subsequent stations (i.e., following a train-oriented perspective). Furthermore, train operations in the railway network involve different types of entities (stations, trains, etc.), making the current graph/network models with homogenous nodes (i.e., the same kind of nodes) incapable of effectively capturing the interactions between the entities. This paper develops a network-oriented model to investigate the train delay evolution on railway networks, by predicting the delays of running trains in the network after a given time interval. The proposed model combines the GraphSAGE graph neural network (GNN) and the heterogeneous graph neural network (HetGNN) architecture, thus called SAGE-Het, enabling it to capture interactions between heterogeneous nodes (i.e., different types of nodes) based on different edges (e.g., edges between trains, trains, and stations). Additionally, SAGE-Het allows for flexible inputs in contrast to conventional machine learning techniques, whose inputs must meet the consistent dimension requirement (e.g., in the form of rectangular or grid-like arrays). The performance and robustness of the suggested SAGE-Het model are assessed in experiments on the data from two sub-networks of the China railway network. The experimental results demonstrate that SAGE-Het outperforms the existing delay prediction methods and some advanced HetGNNs used for prediction tasks in other domains; SAGE-Het demonstrates excellent scalability, capable of handling various types of nodes; the predictive performances of SAGE-Het under different prediction time horizons (10/20/30 min ahead) all exhibit better performance over other baselines; the accuracies are over 90 % under the permissible 3-minute errors for the three prediction time horizons. Specifically, the impact of train interactions on delay evolution is investigated based on the flexible input characteristic of the proposed model. The results illustrate that train interactions become subtle with the increase of train headways. This finding directly contributes to decision-making in situations where conflict resolution or train-canceling actions are needed.

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

In response to the challenges of dynamic adaptability, real-time interactivity, and dynamic optimization posed by the application of existing deep reinforcement learning algorithms in solving complex scheduling problems, this study proposes a novel approach using graph neural networks and deep reinforcement learning to complete the task of job shop scheduling. A distributed multi-agent scheduling architecture (DMASA) is constructed to maximize global rewards, modeling the intelligent manufacturing job shop scheduling problem as a sequential decision problem represented by graphs and using a Graph Embedding–Heterogeneous Graph Neural Network (GE-HetGNN) to encode state nodes and map them to the optimal scheduling strategy, including machine matching and process selection strategies. Finally, an actor–critic architecture-based multi-agent proximal policy optimization algorithm is employed to train the network and optimize the decision-making process. Experimental results demonstrate that the proposed framework exhibits generalizability, outperforms commonly used scheduling rules and RL-based scheduling methods on benchmarks, shows better stability than single-agent scheduling architectures, and breaks through the instance-size constraint, making it suitable for large-scale problems. We verified the feasibility of our proposed method in a specific experimental environment. The experimental results demonstrate that our research can achieve formal modeling and mapping with specific physical processing workshops, which aligns more closely with real-world green scheduling issues and makes it easier for subsequent researchers to integrate algorithms with actual environments.

DialGNN: Heterogeneous Graph Neural Networks for Dialogue Classification

Dialogue systems have attracted growing research interests due to its widespread applications in various domains. However, most research work focus on sentence-level intent recognition to interpret user utterances in dialogue systems, while the comprehension of the whole documents has not attracted sufficient attention. In this paper, we propose DialGNN, a heterogeneous graph neural network framework tailored for the problem of dialogue classification which takes the entire dialogue as input. Specifically, a heterogeneous graph is constructed with nodes in different levels of semantic granularity. The graph framework allows flexible integration of various pre-trained language representation models, such as BERT and its variants, which endows DialGNN with powerful text representational capabilities. DialGNN outperforms on CM and ECS datasets, which demonstrates robustness and the effectiveness. Specifically, our model achieves a notable enhancement in performance, optimizing the classification of document-level dialogue text. The implementation of DialGNN and related data are shared through https://github.com/821code/DialGNN.

Proactive Return Prediction in Online Fashion Retail Using Heterogeneous Graph Neural Networks

Online fashion retailers face enormous challenges due to high return rates that significantly affect their operational performance. Proactively predicting returns at the point of order placement allows for preemptive interventions to reduce potentially problematic transactions. We propose an innovative inductive Heterogeneous Graph Neural Network tailored for proactive return prediction within the realm of online fashion retail. Our model intricately encapsulates customer preferences, product attributes, and order characteristics, providing a holistic approach to return prediction. Through evaluation using real-world data sourced from an online fashion retail platform, our methodology demonstrates superior predictive accuracy on the return behavior of repeat customers, compared to conventional machine learning techniques. Furthermore, through ablation analysis, we underscore the importance of simultaneously capturing customer, order, and product characteristics for an effective proactive return prediction model.

TemporalHAN: Hierarchical attention-based heterogeneous temporal network embedding

Heterogeneous temporal network embedding aims to learn each node of different types of a heterogeneous temporal network in each snapshot into a low-dimensional vector representation, which can be used for various network analysis tasks such as node classification and relationship prediction. Our work proposes a novel heterogeneous temporal graph neural network embedding framework (TemporalHAN) based on hierarchical attention using a temporal convolutional network (TCN). In particular, we introduce node-level and semantic-level attention into heterogeneous graph neural networks to identify the importance of different levels between nodes. For each snapshot, we first utilise a new random walk algorithm (NRWA) to collect strongly connected heterogeneous neighbours for each node of different types and group them by node types. In addition, the algorithm utilises a damping factor to ensure that the more recent snapshots allocate more random walk steps. We then utilise node-level and semantic-level attention to learn the importance between a node and its random walk neighbour for a specific node type and learn the importance of different node-type for this node, respectively. Finally, we adopt TCN to capture the evolution information between snapshots. Experimental results on relationship prediction and node classification reveal that the TemporalHAN is competitive against diverse state-of-the-art approaches. Our code is available at https://github.com/Legendary-L/THAN.

Interpretable answer retrieval based on heterogeneous network embedding

Community question answering is a rising technology based on users' autonomous interactive behaviors, such as posting their issues, answering questions based on their experience, and commenting on existing questions. As a result of its use of natural language for communication and stimulation of user interest in information sharing, it has increasingly taken the place of other channels as the main way that people learn new things. Multi-type entity characteristics fusion and poor answer interpretability are the two major concerns that currently plague community answer prediction research. The Interpretable Answer Retrieval Method Based on Heterogeneous Network Embedding (IARHNE) is what we present in this work. It combines complex entity features and generates interpretable predicted answers. In order to incorporate the interactions of several kinds of individuals in answer social retrieval, we first build a heterogeneity graph. In order to acquire entity embeddings, we secondly use the heterogeneous graph neural network. We then adopt the vector distance to convert the entity matching problem in the heterogeneous information network into a homogeneous node similarity job. Finally, using entity correlation to predict answers, we provide a list of answers to the new query and interpret them using meta-paths. Comparative studies using three authentic datasets demonstrate the benefits of IARHNE for interpretative question-answering research.

Meta-DHGNN: method for CRS-related cytokines analysis in CAR-T therapy based on meta-learning directed heterogeneous graph neural network.

Chimeric antigen receptor T-cell (CAR-T) immunotherapy, a novel approach for treating blood cancer, is associated with the production of cytokine release syndrome (CRS), which poses significant safety concerns for patients. Currently, there is limited knowledge regarding CRS-related cytokines and the intricate relationship between cytokines and cells. Therefore, it is imperative to explore a reliable and efficient computational method to identify cytokines associated with CRS. In this study, we propose Meta-DHGNN, a directed and heterogeneous graph neural network analysis method based on meta-learning. The proposed method integrates both directed and heterogeneous algorithms, while the meta-learning module effectively addresses the issue of limited data availability. This approach enables comprehensive analysis of the cytokine network and accurate prediction of CRS-related cytokines. Firstly, to tackle the challenge posed by small datasets, a pre-training phase is conducted using the meta-learning module. Consequently, the directed algorithm constructs an adjacency matrix that accurately captures potential relationships in a more realistic manner. Ultimately, the heterogeneous algorithm employs meta-photographs and multi-head attention mechanisms to enhance the realism and accuracy of predicting cytokine information associated with positive labels. Our experimental verification on the dataset demonstrates that Meta-DHGNN achieves favorable outcomes. Furthermore, based on the predicted results, we have explored the multifaceted formation mechanism of CRS in CAR-T therapy from various perspectives and identified several cytokines, such as IFNG (IFN-γ), IFNA1, IFNB1, IFNA13, IFNA2, IFNAR1, IFNAR2, IFNGR1 and IFNGR2 that have been relatively overlooked in previous studies but potentially play pivotal roles. The significance of Meta-DHGNN lies in its ability to analyze directed and heterogeneous networks in biology effectively while also facilitating CRS risk prediction in CAR-T therapy.

Heterogeneous graph neural network with graph-data augmentation and adaptive denoising

Heterogeneous graph neural network with graph-data augmentation and adaptive denoising

An Efficient Subgraph-Inferring Framework for Large-Scale Heterogeneous Graphs

Heterogeneous Graph Neural Networks (HGNNs) play a vital role in advancing the field of graph representation learning by addressing the complexities arising from diverse data types and interconnected relationships in real-world scenarios. However, traditional HGNNs face challenges when applied to large-scale graphs due to the necessity of training or inferring on the entire graph. As the size of the heterogeneous graphs increases, the time and memory overhead required by these models escalates rapidly, even reaching unacceptable levels. To address this issue, in this paper, we present a novel framework named (SubInfer), which conducts training and inferring on subgraphs instead of the entire graphs, hence efficiently handling large-scale heterogeneous graphs. The proposed framework comprises three main steps: 1) partitioning the heterogeneous graph from multiple perspectives to preserve various semantic information, 2) completing the subgraphs to improve the convergence speed of subgraph training and the performance of subgraph inference, and 3) training and inferring the HGNN model on distributed clusters to further reduce the time overhead. The framework is applicable to the vast majority of HGNN models. Experiments on five benchmark datasets demonstrate that SubInfer effectively optimizes the training and inference phase, delivering comparable performance to traditional HGNN models while significantly reducing time and memory overhead.

HGPrompt: Bridging Homogeneous and Heterogeneous Graphs for Few-Shot Prompt Learning

Graph neural networks (GNNs) and heterogeneous graph neural networks (HGNNs) are prominent techniques for homogeneous and heterogeneous graph representation learning, yet their performance in an end-to-end supervised framework greatly depends on the availability of task-specific supervision. To reduce the labeling cost, pre-training on self-supervised pretext tasks has become a popular paradigm, but there is often a gap between the pre-trained model and downstream tasks, stemming from the divergence in their objectives. To bridge the gap, prompt learning has risen as a promising direction especially in few-shot settings, without the need to fully fine-tune the pre-trained model. While there has been some early exploration of prompt-based learning on graphs, they primarily deal with homogeneous graphs, ignoring the heterogeneous graphs that are prevalent in downstream applications. In this paper, we propose HGPROMPT, a novel pre-training and prompting framework to unify not only pre-training and downstream tasks but also homogeneous and heterogeneous graphs via a dual-template design. Moreover, we propose dual-prompt in HGPROMPT to assist a downstream task in locating the most relevant prior to bridge the gaps caused by not only feature variations but also heterogeneity differences across tasks. Finally, we thoroughly evaluate and analyze HGPROMPT through extensive experiments on three public datasets.

Abstract 902: Explainable AI: Graph machine learning for response prediction and biomarker discovery

Abstract Accurately predicting drug sensitivity and understanding what is driving it are major challenges in drug discovery. Graphs are a natural framework for capturing diverse pharmacological data for efficacy predictions, thanks to their ability to integrate multimodal data and represent relationships such as gene-gene or drug-target interactions as edges. They have also had proven success across a range of other drug discovery tasks including repositioning and target identification. In this study, we sought to address the explainability challenges of drug response predictions. Recent developments in the field of Graph AI have led to improvements in interpretability mechanisms that highlight parts of a graph which are driving predictions. We have conducted a comprehensive review of multiple major approaches for tackling drug efficacy prediction using graph methods, benchmarking the performance and interpretability of these algorithms across indications. Methods: We assembled a combined dataset of GDSC1 and GDSC2 drug response data in cell lines, with multiomic cell line data and drug target and chemical structure data. We then applied graph-based approaches for the prediction of binarized IC50 on an indication-by-indication basis. Approach 1 involved the creation of a ‘GDSC knowledge graph’, where drug response and cell line ‘omic information is represented in an unweighted knowledge graph: cell lines are connected to genes expressed in them, drugs are connected to genes they target, and so on. We then used state-of-the-art graph embedding techniques to predict IC50 using paired drug and cell line embeddings. In Approach 2 we used a weighted knowledge graph instead, and generated embeddings using heterogeneous graph neural networks (HGNNs). In Approach 3, we modelled response prediction as a graph classification task, where one single graph captures one drug-cell line interaction. The graph classifier and HGNN models both have in-built interpretability mechanisms, including graph attention, that can signify the genes in the cell line which were most important for the eventual prediction. We can also integrate biomedical prior knowledge with all these models by capturing gene-pathway and gene-gene data in the graphs. Results: Our models outperformed benchmark models including DNNs and GBMs, and identified both established and novel response biomarkers in NSCLC cell lines (AUC = 0.94, Accuracy = 89%). We have also applied our models to Breast Cancer, Pancreatic Cancer, Colorectal Cancer and Haematological malignancies with similar predictive performance and explainability. Conclusions:Our graph analytical framework for response predictions showed better performance than benchmarking models and provided insights from explainability. This framework is easily extendable to response and ‘omic data from any disease model and patient studies. Citation Format: Jake Cohen-Setton, Krishna Bulusu, Jonathan Dry, Ben Sidders. Explainable AI: Graph machine learning for response prediction and biomarker discovery [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 902.

A graph neural network with topic relation heterogeneous multi-level cross-item information for session-based recommendation

The aim of session-based recommendation (SBR) mainly analyzes the anonymous user’s historical behavior records to predict the next possible interaction item and recommend the result to the user. However, due to the anonymity of users and the sparsity of behavior records, recommendation results are often inaccurate. The existing SBR models mainly consider the order of items within a session and rarely analyze the complex transition relationship between items, and additionally, they are inadequate at mining higher-order hidden relationship between different sessions. To address these issues, we propose a topic relation heterogeneous multi-level cross-item information graph neural network (TRHMCI-GNN) to improve the performance of recommendation. The model attempts to capture hidden relationship between items through topic classification and build a topic relation heterogeneous cross-item global graph. The graph contains inter-session cross-item information as well as hidden topic relation among sessions. In addition, a self-loop star graph is established to learn the intra-session cross-item information, and the self-connection attributes are added to fuse the information of each item itself. By using channel-hybrid attention mechanism, the item information of different levels is pooled by two channels: max-pooling and mean-pooling, which effectively fuse the item information of cross-item global graph and self-loop star graph. In this way, the model captures the global information of the target item and its individual features, and the label smoothing operation is added for recommendation. Extensive experimental results demonstrate that the recommendation performance of TRHMCI-GNN model is superior to the comparable baseline models on the three real datasets Diginetica, Yoochoose1/64 and Tmall. The code is available now.11https://github.com/usstyangfan/TRHMCI-GNN.

Multipath-guided heterogeneous graph neural networks for sequential recommendation

With the explosion of information and users’ changing interest, program sequential recommendation becomes increasingly important for TV program platforms to help their users find interesting programs. Existing sequential recommendation methods mainly focus on modeling user preferences from users’ historical interaction behaviors directly, with insufficient learning about the dynamics of programs and users, while ignoring the rich semantic information from the heterogeneous graph. To address these issues, we propose the multipath-guided heterogeneous graph neural networks for TV program sequential recommendation (MHG-PSR), which can enhance the representations of programs and users through multiple paths in heterogeneous graphs. In our method, the auxiliary information is fused to supplement the semantics of program and user to obtain initial representations. Then, we explore the interactive behaviors of programs and users with temporal and auxiliary information to model the collaborative signals in the heterogeneous graph and extract the users’ dynamic preferences of programs. Extensive experiments on real-world datasets verify the proposed method can effectively improve the performance of TV program sequential recommendation.

Heterogeneous Graph Neural Network with Personalized and Adaptive Diversity for News Recommendation

The emergence of online media has facilitated the dissemination of news, but has also introduced the problem of information overload. To address this issue, providing users with accurate and diverse news recommendations has become increasingly important. News possesses rich and heterogeneous content, and the factors that attract users to news reading are varied. Consequently, accurate news recommendation requires modeling of both the heterogeneous content of news and the heterogeneous user-news relationships. Furthermore, users’ news consumption is highly dynamic, which is reflected in the differences in topic concentration among different users and in the real-time changes in user interests. To this end, we propose a Heterogeneous Graph Neural Network with Personalized and Adaptive Diversity for News Recommendation (DivHGNN). DivHGNN first represents the heterogeneous content of news and the heterogeneous user-news relationships as an attributed heterogeneous graph. Then, through a heterogeneous node content adapter, it models the heterogeneous node attributes into aligned and fused node representations. With the proposed attributed heterogeneous graph neural network, DivHGNN integrates the heterogeneous relationships to enhance node representation for accurate news recommendations. We also discuss relation pruning, model deployment, and cold-start issues to further improve model efficiency. In terms of diversity, DivHGNN simultaneously models the variance of nodes through variational representation learning for providing personalized diversity. Additionally, a time-continuous exponentially decaying distribution cache is proposed to model the temporal dynamics of user real-time interests for providing adaptive diversity. Extensive experiments on real-world news datasets demonstrate the effectiveness of the proposed method.

Node classification oriented Adaptive Multichannel Heterogeneous Graph Neural Network

Heterogeneous graph neural networks (HGNNs) play an important role in accomplishing node classification on heterogeneous graphs (HGs). These models are built on the traditional spatial graph neural networks (GNNs) framework of neighborhood sampling, message passing, and aggregation. However, like GNNs, HGNNs face challenges in capturing high-order neighbor information without oversmoothing or classifying vertices with flexible topologies. To address these issues, we propose a novel Adaptive Multichannel Heterogeneous Graph Neural Network (AMHGNN) that adaptively utilizes High and Low-frequency signals, resulting in a more accurate node classification with flexible topologies. Specifically, we first identify the vertices with the closest topological associations for each vertex using a Markov process, regardless of whether they are originally connected, and add them to the original HG as a new type of edge that directly captures the information from relevant arbitrary-order neighbors without oversmoothing. Second, we designed frequency-adaptive heterogeneous graph neural networks that map vertices and edges of different types to the same vector space, separate high- and low-frequency signals, and adaptively aggregate them for each vertex. Extensive experiments on four real-world HGs demonstrated that AMHGNN achieved excellent performance across various types of datasets, particularly those that demand more high-frequency signals. The code is available at https://github.com/LIyvqi/AMHGNN.