Heterogeneous Graph Research Articles

A representation learning-based approach to enhancing manufacturing quality for low-voltage electrical products

In low-voltage electrical product manufacturing, resolving quality issues is heavily reliant on engineering experience, and can be time-consuming and error-prone. Through quality management systems, a large number of historical defect cases can be consolidated for analysis along with relevant causes. However, these defect descriptions are often casually described with a mix of Chinese and English language, containing domain-specific terms. Additionally, defect product features have varieties and complex relationships. Therefore, historical defect cases have not been effectively utilized to support manufacturing quality issues. To address this challenge, this study proposes a representation learning-based approach to enhance manufacturing quality. Key research contributions include: (1) A two-stage word embedding technique based on the pre-trained language model. First, TSDAE is utilized for unsupervised pre-training on a large amount of unlabeled data. Then, Sentence-BERT is utilized for fine-tuning on a small set of labeled similar sentence pairs. This process yields a pre-trained language model specific to low-voltage electrical product defects. (2) NSHPSAGE graph embedding model based on the constructed product feature knowledge graph. We select more valuable neighboring nodes during sampling and explore different aggregation functions to enhance graph embedding performance. This model effectively aggregates product feature information into “Defect_Case” nodes, yielding graph embedding vectors. The model exhibits good Weighted-Precision and Weighted-Recall with a short training duration, and it can handle new nodes, addressing the issue of heterogeneous graph embedding. (3) A defect case recommendation technique that fuses word embedding and graph embedding. We use Multi-Head Attention Fusion in the late-fusion to obtain defect case vectors. This approach comprehensively considers defect description semantic knowledge and complex product feature relationships, enabling accurate defect case recommendation with the prototype system.

GABoost: Graph Alignment Boosting via Local Optimum Escape

Heterogeneous graphs provide a universal data structure for representing various kinds of structured data in numerous domains. The graph alignment problem aims to find the correspondences of vertices in different graphs, playing a fundamental role in many downstream tasks of heterogeneous graph mining. In recent years, many graph alignment methods have been proposed, ranging from classical optimization methods , spectral methods , to embedding learning based-methods . Due to the problem's complexity, the result found by most existing methods is either a heuristic solution or a critical point in the solution space. In this paper, we propose GABoost, a graph alignment boosting algorithm that takes as input an initial alignment between two heterogeneous graphs and outputs a boosted alignment via an iterative local-optimum-escape process. One of the distinctive features of GABoost is that it can be sequentially composed with any graph alignment methods to improve the output of upstream methods. To examine the effectiveness of GABoost, we select 7 upstream methods of graph alignment as well as 6 real-world datasets, and quantitatively investigate the degree to which GABoost boosts these methods. The results show that GABoost improves the alignment accuracy of the 7 upstream methods by 25.25% on average with acceptable time overhead.

Finding meaningful paths in heterogeneous graphs with PathWays

Graphs, and notably RDF graphs, are a prominent way of sharing data. As data usage democratizes, users need help figuring out the useful content of a graph dataset. In particular, journalists with whom we collaborate are interested in identifying, in a graph, the connections between entities, e.g., people, organizations, emails, etc. We present a novel method for exploring data graphs through their data paths connecting Named Entities (NEs, in short); each data path leads to a tabular-looking set of results. NEs are extracted from the data through dedicated Information Extraction modules. Our method builds upon the pre-existing ConnectionLens platform and follow-up work in the Abstra project, which builds simple, visual ER-style summaries of semi-structured data. The contribution of the present work, and its novelty, is twofold. First, we propose a novel analysis of entity-to-entity paths contained in datasets of any nature, and propose a new method for ranking paths, leveraging a novel Information Extraction (IE) module we built on top of ChatGPT. Second, we present an efficient approach to enumerate and compute NE paths, based on an algorithm which automatically recommends sub-paths to materialize, and rewrites the path queries using these subpaths. Our experiments demonstrate the interest of NE paths and the efficiency of our method for computing and ranking them.

Trust enhanced POI recommendation with collaborative learning

With the rapid development and popularization of smart mobile devices, users tend to share their visited points-of-interest (POIs) on the network with attached location information, which forms a location-based social network (LBSN). LBSNs contain a wealth of valuable information, including the geographical coordinates of POIs and the social connections among users. Nowadays, lots of trust-enhanced approaches have fused the trust relationships of users together with other auxiliary information to provide more accurate recommendations. However, in the traditional trust-aware approaches, the embedding processes of the information on different graphs with different properties (e.g., user-user graph is an isomorphic graph, user-POI graph is a heterogeneous graph) are independent of each other and different embedding information is directly fused together without guidance, which limits their performance. More effective information fusion strategies are needed to improve the performance of trust-enhanced recommendation. To this end, we propose a Trust Enhanced POI recommendation approach with Collaborative Learning (TECL) to merge geographic information and social influence. Our proposed model integrates two modules, a GAT-based graph autoencoder as trust relationships embedding module and a multi-layer deep neural network as a user-POI graph learning module. By applying collaborative learning strategy, these two modules can interact with each other. The trust embedding module can guide the selection of user’s potential features, and in turn the user-POI graph learning module enhances the embedding process of trust relationships. Different information is fused through the two-way interaction of information, instead of travelling in one direction. Extensive experiments are conducted using real-world datasets, and results illustrate that our suggested approach outperforms state-of-the-art methods.

Fine-grained multimodal named entity recognition with heterogeneous image-text similarity graphs

Fine-grained multimodal named entity recognition with heterogeneous image-text similarity graphs

ACLNDA: an asymmetric graph contrastive learning framework for predicting noncoding RNA-disease associations in heterogeneous graphs.

Noncoding RNAs (ncRNAs), including long noncoding RNAs (lncRNAs) and microRNAs (miRNAs), play crucial roles in gene expression regulation and are significant in disease associations and medical research. Accurate ncRNA-disease association prediction is essential for understanding disease mechanisms and developing treatments. Existing methods often focus on single tasks like lncRNA-disease associations (LDAs), miRNA-disease associations (MDAs), or lncRNA-miRNA interactions (LMIs), and fail to exploit heterogeneous graph characteristics. We propose ACLNDA, an asymmetric graph contrastive learning framework for analyzing heterophilic ncRNA-disease associations. It constructs inter-layer adjacency matrices from the original lncRNA, miRNA, and disease associations, and uses a Top-K intra-layer similarity edges construction approach to form a triple-layer heterogeneous graph. Unlike traditional works, to account for both node attribute features (ncRNA/disease) and node preference features (association), ACLNDA employs an asymmetric yet simple graph contrastive learning framework to maximize one-hop neighborhood context and two-hop similarity, extracting ncRNA-disease features without relying on graph augmentations or homophily assumptions, reducing computational cost while preserving data integrity. Our framework is capable of being applied to a universal range of potential LDA, MDA, and LMI association predictions. Further experimental results demonstrate superior performance to other existing state-of-the-art baseline methods, which shows its potential for providing insights into disease diagnosis and therapeutic target identification. The source code and data of ACLNDA is publicly available at https://github.com/AI4Bread/ACLNDA.

Structure-preserved integration of scRNA-seq data using heterogeneous graph neural network.

The integration of single-cell RNA sequencing (scRNA-seq) data from multiple experimental batches enables more comprehensive characterizations of cell states. Given that existing methods disregard the structural information between cells and genes, we proposed a structure-preserved scRNA-seq data integration approach using heterogeneous graph neural network (scHetG). By establishing a heterogeneous graph that represents the interactions between multiple batches of cells and genes, and combining a heterogeneous graph neural network with contrastive learning, scHetG concurrently obtained cell and gene embeddings with structural information. A comprehensive assessment covering different species, tissues and scales indicated that scHetG is an efficacious method for eliminating batch effects while preserving the structural information of cells and genes, including batch-specific cell types and cell-type specific gene co-expression patterns.

Modeling Brain Functional Networks Using Graph Neural Networks: A Review and Clinical Application

The integration of graph neural networks (GNNs) with brain functional network analysis is an emerging field that combines neuroscience and machine learning to enhance our understanding of complex brain dynamics. We first briefly introduce the fundamentals of brain functional networks, followed by an overview of Graph Neural Network principles and architectures. The review then focuses on the applications of these networks and address current challenges in the field, such as the need for interpretable models and effective integration of multi-modal neuroimaging data. We also highlight the potential of GNNs in clinical perimenopausal areas such as perimenopausal depression research, demonstrating the broad applicability of this approach. The review concludes by outlining future research directions, including the development of more sophisticated architectures for large-scale, heterogeneous brain graphs, and the exploration of causal inference in brain networks. By synthesizing recent advances and identifying key research directions, this review aims to summarize the focal points of brain functional network analysis and GNNs, explore the potential of their integration, and provide a reference for advancing this interdisciplinary field.

CrossFormer: Cross-modal Representation Learning via Heterogeneous Graph Transformer

CrossFormer: Cross-modal Representation Learning via Heterogeneous Graph Transformer

Graph contrast learning for recommendation based on relational graph convolutional neural network

Current knowledge graph-based recommendation methods heavily rely on high-quality knowledge graphs, often falling short in effectively addressing issues such as the cold start problem and heterogeneous noise in user interactions. This leads to biases in user interest and popularity. To overcome these challenges, this paper introduces a novel recommendation approach termed Knowledge-enhanced Perceptive Graph Attention with Graph Contrastive Learning (KPA-GCL), which leverages relational graph convolutional neural networks. The proposed method optimizes the triplet embedding representation of entity-item interactions based on relationships between adjacent entities in a heterogeneous graph. Subsequently, a graph convolutional neural network is employed for enhanced aggregation. Similarity scores from a contrastive view serve as the selection criterion for high-quality embedded representations, facilitating the extraction of refined knowledge subgraphs. Multiple adaptive contrast-loss optimization functions are introduced by combining Bayesian Personalized Ranking (BPR) and hard negative sampling techniques. Comparative experiments are conducted with ten popular existing methods using real public datasets. Results indicate that the KPA-GCL method outperforms compared methods in all datasets based on Recall, NDCG, Precision, and Hit-ratio measures. Furthermore, in terms of mitigating cold start and noise, the KPA-GCL method surpasses other ten methods. This validates the reasonability and effectiveness of KPA-GCL in real-world datasets.

An Explainable Graph Neural Framework to Identify Cancer-Associated Intratumoral Microbial Communities.

Microbes are extensively present among various cancer tissues and play critical roles in carcinogenesis and treatment responses. However, the underlying relationships between intratumoral microbes and tumors remain poorly understood. Here, a MIcrobial Cancer-association Analysis using a Heterogeneous graph transformer (MICAH) to identify intratumoral cancer-associated microbial communities is presented. MICAH integrates metabolic and phylogenetic relationships among microbes into a heterogeneous graph representation. It uses a graph transformer to holistically capture relationships between intratumoral microbes and cancer tissues, which improves the explainability of the associations between identified microbial communities and cancers. MICAH is applied to intratumoral bacterial data across 5 cancer types and 5 fungi datasets, and its generalizability and reproducibility are demonstrated. After experimentally testing a representative observation using a mouse model of tumor-microbe-immune interactions, a result consistent with MICAH's identified relationship is observed. Source tracking analysis reveals that the primary known contributor to a cancer-associated microbial community is the organs affected by the type of cancer. Overall, this graph neural network framework refines the number of microbes that can be used for follow-up experimental validation from thousands to tens, thereby helping to accelerate the understanding of the relationship between tumors and intratumoral microbiomes.

A method for miRNA diffusion association prediction using machine learning decoding of multi-level heterogeneous graph Transformer encoded representations

MicroRNAs (miRNAs) are a key class of endogenous non-coding RNAs that play a pivotal role in regulating diseases. Accurately predicting the intricate relationships between miRNAs and diseases carries profound implications for disease diagnosis, treatment, and prevention. However, these prediction tasks are highly challenging due to the complexity of the underlying relationships. While numerous effective prediction models exist for validating these associations, they often encounter information distortion due to limitations in efficiently retaining information during the encoding-decoding process. Inspired by Multi-layer Heterogeneous Graph Transformer and Machine Learning XGboost classifier algorithm, this study introduces a novel computational approach based on multi-layer heterogeneous encoder—machine learning decoder structure for miRNA-disease association prediction (MHXGMDA). First, we employ the multi-view similarity matrices as the input coding for MHXGMDA. Subsequently, we utilize the multi-layer heterogeneous encoder to capture the embeddings of miRNAs and diseases, aiming to capture the maximum amount of relevant features. Finally, the information from all layers is concatenated to serve as input to the machine learning classifier, ensuring maximal preservation of encoding details. We conducted a comprehensive comparison of seven different classifier models and ultimately selected the XGBoost algorithm as the decoder. This algorithm leverages miRNA embedding features and disease embedding features to decode and predict the association scores between miRNAs and diseases. We applied MHXGMDA to predict human miRNA-disease associations on two benchmark datasets. Experimental findings demonstrate that our approach surpasses several leading methods in terms of both the area under the receiver operating characteristic curve and the area under the precision-recall curve.

Enhancing urban functional regions classification by integrating multi-source data with a heterogeneous graph neural network

Enhancing urban functional regions classification by integrating multi-source data with a heterogeneous graph neural network

NLA-GNN: Non-local information aggregated graph neural network for heterogeneous graph embedding

Heterogeneous graphs are ubiquitous in the real world. Recent methods aim to obtain meaningful low-dimensional node embeddings from heterogeneous graphs to facilitate downstream applications. However, most existing methods tend to consider the local information but ignore the non-local information. This paper proposes a novel Non-Local Information Aggregated Graph Neural Network (NLA-GNN) that aggregate not only the local information from neighbor nodes but also non-local information from distant nodes. Specifically, Local aggregation modules in NLA-GNN utilize the attention mechanism to generate potentially valuable metapaths and use them to aggregate local information. In contrast, non-local aggregation modules adopt a two-step approach, and each step uses an attention-guided method to sort nodes into node sequences and aggregate information with the methods designed for sequential data. Experiment results on three heterogeneous graph datasets demonstrate the performance of NLA-GNN over state-of-the-arts and the necessity of non-local aggregation in heterogeneous graphs.

Dynamic category-sensitive hypergraph inferring and homo-heterogeneous neighbor feature learning for drug-related microbe prediction.

The microbes in human body play a crucial role in influencing the functions of drugs, as they can regulate the activities and toxicities of drugs. Most recent methods for predicting drug-microbe associations are based on graph learning. However, the relationships among multiple drugs and microbes are complex, diverse, and heterogeneous. Existing methods often fail to fully model the relationships. In addition, the attributes of drug-microbe pairs exhibit long-distance spatial correlations, which previous methods have not integrated effectively. We propose a new prediction method named DHDMP which is designed to encode the relationships among multiple drugs and microbes and integrate the attributes of various neighbor nodes along with the pairwise long-distance correlations. First, we construct a hypergraph with dynamic topology, where each hyperedge represents a specific relationship among multiple drug nodes and microbe nodes. Considering the heterogeneity of node attributes across different categories, we developed a node category-sensitive hypergraph convolution network to encode these diverse relationships. Second, we construct homogeneous graphs for drugs and microbes respectively, as well as drug-microbe heterogeneous graph, facilitating the integration of features from both homogeneous and heterogeneous neighbors of each target node. Third, we introduce a graph convolutional network with cross-graph feature propagation ability to transfer node features from homogeneous to heterogeneous graphs for enhanced neighbor feature representation learning. The propagation strategy aids in the deep fusion of features from both types of neighbors. Finally, we design spatial cross-attention to encode the attributes of drug-microbe pairs, revealing long-distance correlations among multiple pairwise attribute patches. The comprehensive comparison experiments showed our method outperformed state-of-the-art methods for drug-microbe association prediction. The ablation studies demonstrated the effectiveness of node category-sensitive hypergraph convolution network, graph convolutional network with cross-graph feature propagation, and spatial cross-attention. Case studies on three drugs further showed DHDMP's potential application in discovering the reliable candidate microbes for the interested drugs. Source codes and supplementary materials are available at https://github.com/pingxuan-hlju/DHDMP.

Temporal Graph Network for continuous-time dynamic event sequence

Continuous-Time Dynamic Graph (CTDG) methods have shown their superior ability in learning representations for dynamic graph-structured data, the methods split the sequential updating process into discrete batches to reduce the computation costs, as a result, the message constructor in existing CTDG methods cannot be optimized by gradient descent and is designed to be parameter-free. In particular, this layer fails to embed complex event subgraphs and ignores the structure information, while most real-world events are structured and complex. For example, a paper publication event in an academic graph contains different relations like authorship and citations. Furthermore, the corresponding nodes could not receive position-wise messages to make precise representation updates. To tackle this issue, we propose a new method called Temporal Graph Network for continuous-time dynamic Event sequence (TGNE) with a structure-aware message constructor to update node representation with complex event subgraph, by treating message construction and delivery as a message-passing process, in this way, the message constructor can be formalized as a graph neural network layer. TGNE extends the input of CTDG methods to subgraphs with complex structures and preserves more information in message delivery. Extensive experiments demonstrate that the proposed method can achieve competitive performance on traditional tasks on bipartite graphs and event sequence learning tasks on heterogeneous graphs.

Contrastive meta-reinforcement learning for heterogeneous graph neural architecture search

Heterogeneous Graph Neural Networks (HGNNs) have demonstrated significant success in capturing complex interactions within heterogeneous graphs to learn graph representations. However, designing effective HGNN architecture is a challenging task that demands substantial domain knowledge and human effort. Fortunately, the advent of Heterogeneous Graph Architecture Search (HGNAS) has automated this process and yielded networks surpassing those designed by humans. Nonetheless, conventional HGNAS approaches are limited in their ability to generalize to unseen or diverse task types. To address these limitations, we propose an efficient method called Contrastive Meta-reinforcement learning-based Heterogeneous Graph Neural Architecture Search (CM-HGNAS). Our approach aims to produce high-performance networks tailored to different types of tasks. Firstly, we use gradient-based meta-learning to rapidly adapt to new tasks by leveraging the knowledge acquired from meta-training tasks. Secondly, since our meta-training tasks encompass various types of tasks, including node classification and link prediction, we employ contrastive learning to unify the evaluation metric across all tasks. This unification enhances the generalization capabilities of our model across diverse tasks. Experimental results demonstrate that our method successfully generalizes to unseen tasks and surpasses existing HGNN baselines and HGNAS techniques in terms of performance.

Diagnosing Performance Issues for Large-Scale Microservice Systems With Heterogeneous Graph

Diagnosing Performance Issues for Large-Scale Microservice Systems With Heterogeneous Graph

DDI Prediction With Heterogeneous Information Network - Meta-Path Based Approach.

Drug-drug interaction (DDI) indicates where a particular drug's desired course of action is modified when taken with other drug (s). DDIs may hamper, enhance, or reduce the expected effect of either drug or, in the worst possible scenario, cause an adverse side effect. While it is crucial to identify drug-drug interactions, it is quite impossible to detect all possible DDIs for a new drug during the clinical trial. Therefore, many computational methods are proposed for this task. This paper presents a novel method based on a heterogeneous information network (HIN), which consists of drugs and other biomedical entities like proteins, pathways, and side effects. Afterward, we extract the rich semantic relationships among these entities using different meta-path-based topological features and facilitate DDI prediction. In addition, we present a heterogeneous graph attention network-based end-to-end model for DDI prediction in the heterogeneous graph. Experimental results show that our proposed method accurately predicts DDIs and outperforms the baselines significantly.

SHINE: A Scalable Heterogeneous Inductive Graph Neural Network for Large Imbalanced Datasets

SHINE: A Scalable Heterogeneous Inductive Graph Neural Network for Large Imbalanced Datasets