Node Embedding Research Articles

Path-based reasoning for biomedical knowledge graphs with BioPathNet.

Understanding complex interactions in biomedical networks is crucial for advancements in biomedicine, but traditional link prediction (LP) methods are limited in capturing this complexity. Representation-based learning techniques improve prediction accuracy by mapping nodes to low-dimensional embeddings, yet they often struggle with interpretability and scalability. We present BioPathNet, a novel graph neural network framework based on the Neural Bellman-Ford Network (NBFNet), addressing these limitations through path-based reasoning for LP in biomedical knowledge graphs. Unlike node-embedding frameworks, BioPathNet learns representations between node pairs by considering all relations along paths, enhancing prediction accuracy and interpretability. This allows visualization of influential paths and facilitates biological validation. BioPathNet leverages a background regulatory graph (BRG) for enhanced message passing and uses stringent negative sampling to improve precision. In evaluations across various LP tasks, such as gene function annotation, drug-disease indication, synthetic lethality, and lncRNA-mRNA interaction prediction, BioPathNet consistently outperformed shallow node embedding methods, relational graph neural networks and task-specific state-of-the-art methods, demonstrating robust performance and versatility. Our study predicts novel drug indications for diseases like acute lymphoblastic leukemia (ALL) and Alzheimer's, validated by medical experts and clinical trials. We also identified new synthetic lethality gene pairs and regulatory interactions involving lncRNAs and target genes, confirmed through literature reviews. BioPathNet's interpretability will enable researchers to trace prediction paths and gain molecular insights, making it a valuable tool for drug discovery, personalized medicine and biology in general.

Drug repositioning based on residual attention network and free multiscale adversarial training

BackgroundConducting traditional wet experiments to guide drug development is an expensive, time-consuming and risky process. Analyzing drug function and repositioning plays a key role in identifying new therapeutic potential of approved drugs and discovering therapeutic approaches for untreated diseases. Exploring drug-disease associations has far-reaching implications for identifying disease pathogenesis and treatment. However, reliable detection of drug-disease relationships via traditional methods is costly and slow. Therefore, investigations into computational methods for predicting drug-disease associations are currently needed.ResultsThis paper presents a novel drug-disease association prediction method, RAFGAE. First, RAFGAE integrates known associations between diseases and drugs into a bipartite network. Second, RAFGAE designs the Re_GAT framework, which includes multilayer graph attention networks (GATs) and two residual networks. The multilayer GATs are utilized for learning the node embeddings, which is achieved by aggregating information from multihop neighbors. The two residual networks are used to alleviate the deep network oversmoothing problem, and an attention mechanism is introduced to combine the node embeddings from different attention layers. Third, two graph autoencoders (GAEs) with collaborative training are constructed to simulate label propagation to predict potential associations. On this basis, free multiscale adversarial training (FMAT) is introduced. FMAT enhances node feature quality through small gradient adversarial perturbation iterations, improving the prediction performance. Finally, tenfold cross-validations on two benchmark datasets show that RAFGAE outperforms current methods. In addition, case studies have confirmed that RAFGAE can detect novel drug-disease associations.ConclusionsThe comprehensive experimental results validate the utility and accuracy of RAFGAE. We believe that this method may serve as an excellent predictor for identifying unobserved disease-drug associations.

FC-HGNN: A heterogeneous graph neural network based on brain functional connectivity for mental disorder identification

Rapid and accurate diagnosis of mental disorders has long been an essential challenge in clinical medicine. Due to the advantage in addressing non-Euclidean structures, graph neural networks have been increasingly used to study brain networks. Among the existing methods, the population graph models have achieved high predictive accuracy by considering intersubject relationships but weak interpretability limits its clinical applicability. The individual graph approach models functional brain networks and identifies abnormal brain regions that cause diseases but has poor accuracy. To address these issues, we propose a heterogeneous graph neural network based on brain functional connectivity (FC-HGNN), which is an end-to-end model with a two-stage process. In the first phase, the brain connectomic graph is used to extract individual brain features. An integrated intrahemispheric and interhemispheric convolutional graph layer is used to learn brain region features, and a local–global dual-channel pooling layer is used to identify biomarkers. In the second stage, a heterogeneous population graph is constructed based on sex and the fusion of imaging and non-imaging data from subjects. The feature embeddings of same-sex and opposite-sex neighbours are learned separately according to a hierarchical feature aggregation approach. Subsequently, they are adaptively fused to generate the final node embedding, which is then utilized for obtaining classification predictions. The cross-validation and transduction learning results show that FC-HGNN achieves state-of-the-art performance in classification prediction experiments using two public datasets. Moreover, FC-HGNN identifies crucial biomarker regions relevant for disease classification, aligning with existing studies and exhibiting outstanding predictive performance on actual clinical data. The code is available at https://github.com/Guuuu11/FC-HGNN_Pytorch.

Attribute graph clustering via transformer and graph attention autoencoder

Graph clustering is a crucial technique for partitioning graph data. Recent research has concentrated on integrating topology and attribute information from attribute graphs to generate node embeddings, which are subsequently clustered using classical algorithms. However, these methods have some limitations, such as insufficient information inheritance in shallow networks or inadequate quality of reconstructed nodes, leading to suboptimal clustering performance. To tackle these challenges, we introduce two normalization techniques within the graph attention autoencoder framework, coupled with an MSE loss, to facilitate node embedding learning. Furthermore, we integrate Transformers into the self-optimization module to refine node embeddings and clustering outcomes. Our model can induce appropriate node embeddings for graph clustering in a shallow network. Our experimental results demonstrate that our proposed approach outperforms the state-of-the-art in graph clustering over multiple benchmark datasets. In particular, we achieved 76.3% accuracy on the Pubmed dataset, an improvement of at least 7% compared to other methods.

PANNER: POS-Aware Nested Named Entity Recognition Through Heterogeneous Graph Neural Network

Nested named entity recognition (Nested NER) in knowledge graph (KG) aims at obtaining all meaningful entities, including nested entities for sentences in longer text region. Those obtained entities are to facilitate downstream applications, such as relation extraction, entity resolution, and coreference resolution. This task, however, is challenging not only because of the demand to detect the boundary of the entity but also due to the complexity of those hierarchically nested entities. Since a substantial amount of work has been made to Flat NER (or Nested NER), a few of them can explicitly acquire the position of the entity and utilize the grammatical construction of text. In this work, we propose PANNER, a POS-aware Nested NER model, to solve all the above issues. Specifically, we first construct a heterogeneous graph by introducing the part-of-speech (POS) information of the word. Second, we design a dilated random walk (DRW) algorithm based on a grammatical path to sample a fixed size of neighbors for each node. Third, we aggregate the message from different types of neighbors through an attention mechanism. Finally, we use a bidirectional decoding module to recognize and categorize all the flat and nested entities based on the node embedding in a layer-wise manner. Our extensive experiments show the effectiveness of PANNER in both flat and nested NER.

Discovering Microbe-disease Associations with Weighted Graph Convolution Networks and Taxonomy Common Tree

Background: Microbe-disease associations are integral to understanding complex diseases and their screening procedures. Objective: While numerous computational methods have been developed to detect these associations, their performance remains limited due to inadequate utilization of weighted inherent similarities and microbial taxonomy hierarchy. To address this limitation, we have introduced WTHMDA (weighted taxonomic heterogeneous network-based microbe-disease association), a novel deep learning framework. Methods: WTHMDA combines a weighted graph convolution network and the microbial taxonomy common tree to predict microbe-disease associations effectively. The framework extracts multiple microbe similarities from the taxonomy common tree, facilitating the construction of a microbe- disease heterogeneous interaction network. Utilizing a weighted DeepWalk algorithm, node embeddings in the network incorporate weight information from the similarities. Subsequently, a deep neural network (DNN) model accurately predicts microbe-disease associations based on this interaction network. Results: Extensive experiments on multiple datasets and case studies demonstrate WTHMDA's superiority over existing approaches, particularly in predicting unknown associations. Conclusion: Our proposed method offers a new strategy for discovering microbe-disease linkages, showcasing remarkable performance and enhancing the feasibility of identifying disease risk.

Polypharmacy side effect prediction based on semi-implicit graph variational auto-encoder.

Polypharmacy, the use of drug combinations, is an effective approach for treating complex diseases, but it increases the risk of adverse effects. To predict novel polypharmacy side effects based on known ones, many computational methods have been proposed. However, most of them generate deterministic low-dimensional embeddings when modeling the latent space of drugs, which cannot effectively capture potential side effect associations between drugs. In this study, we present SIPSE, a novel approach for predicting polypharmacy side effects. SIPSE integrates single-drug side effect information and drug-target protein data to construct novel drug feature vectors. Leveraging a semi-implicit graph variational auto-encoder, SIPSE models known polypharmacy side effects and generates flexible latent distributions for drug nodes. SIPSE infers the current node distribution by combining the distributions of neighboring nodes with embedding noise. By sampling node embeddings from these distributions, SIPSE effectively predicts polypharmacy side effects between drugs. One key innovation of SIPSE is its incorporation of uncertainty propagation through noise embedding and neighborhood sharing, enhancing its graph analysis capabilities. Extensive experiments on a benchmark dataset of polypharmacy side effects demonstrated that SIPSE significantly outperformed five state-of-the-art methods in predicting polypharmacy side effects.

CAG-NSPDE: Continuous adaptive graph neural stochastic partial differential equations for traffic flow forecasting

Traffic flow forecasting, which aims to predict future traffic flow given historical data, is crucial for the planning and construction of urban transportation infrastructure. Existing methods based on discrete dynamic graphs are unable to model the smooth evolution of spatial relationships and usually have high computational cost. In addition, they ignore the effect of stochastic noise on the modeling of temporal dependencies, which lead to poor predictive performance on complex intelligent transportation systems. To address these issues, we propose continuous adaptive graph neural stochastic partial differential equation (CAG-NSPDE). A continuous graph is constructed based on learnable continuous adaptive node embeddings for the modeling of evolution of spatial dependencies. To explicitly model the effect of noise, a novel architecture based on stochastic partial differential equation (SPDE) is proposed. It can simultaneously extract features from the temporal and frequency domains by reducing SPDEs to a system of ODEs in Fourier space. We perform extensive experiments on four real-world datasets and the results demonstrate the superiority of CAG-NSPDE compared with state-of-the-arts.

Global-local aware Heterogeneous Graph Contrastive Learning for multifaceted association prediction in miRNA-gene-disease networks.

Unraveling the intricate network of associations among microRNAs (miRNAs), genes, and diseases is pivotal for deciphering molecular mechanisms, refining disease diagnosis, and crafting targeted therapies. Computational strategies, leveraging link prediction within biological graphs, present a cost-efficient alternative to high-cost empirical assays. However, while plenty of methods excel at predicting specific associations, such as miRNA-disease associations (MDAs), miRNA-target interactions (MTIs), and disease-gene associations (DGAs), a holistic approach harnessing diverse data sources for multifaceted association prediction remains largely unexplored. The limited availability of high-quality data, as vitro experiments to comprehensively confirm associations are often expensive and time-consuming, results in a sparse and noisy heterogeneous graph, hindering an accurate prediction of these complex associations. To address this challenge, we propose a novel framework called Global-local aware Heterogeneous Graph Contrastive Learning (GlaHGCL). GlaHGCL combines global and local contrastive learning to improve node embeddings in the heterogeneous graph. In particular, global contrastive learning enhances the robustness of node embeddings against noise by aligning global representations of the original graph and its augmented counterpart. Local contrastive learning enforces representation consistency between functionally similar or connected nodes across diverse data sources, effectively leveraging data heterogeneity and mitigating the issue of data scarcity. The refined node representations are applied to downstream tasks, such as MDA, MTI, and DGA prediction. Experiments show GlaHGCL outperforming state-of-the-art methods, and case studies further demonstrate its ability to accurately uncover new associations among miRNAs, genes, and diseases. We have made the datasets and source code publicly available at https://github.com/Sue-syx/GlaHGCL.

ScHyper: reconstructing cell-cell communication through hypergraph neural networks.

Cell-cell communications is crucial for the regulation of cellular life and the establishment of cellular relationships. Most approaches of inferring intercellular communications from single-cell RNA sequencing (scRNA-seq) data lack a comprehensive global network view of multilayered communications. In this context, we propose scHyper, a new method that can infer intercellular communications from a global network perspective and identify the potential impact of all cells, ligand, and receptor expression on the communication score. scHyper designed a new way to represent tripartite relationships, by extracting a heterogeneous hypergraph that includes the source (ligand expression), the target (receptor expression), and the relevant ligand-receptor (L-R) pairs. scHyper is based on hypergraph representation learning, which measures the degree of match between the intrinsic attributes (static embeddings) of nodes and their observed behaviors (dynamic embeddings) in the context (hyperedges), quantifies the probability of forming hyperedges, and thus reconstructs the cell-cell communication score. Additionally, to effectively mine the key mechanisms of signal transmission, we collect a rich dataset of multisubunit complex L-R pairs and propose a nonparametric test to determine significant intercellular communications. Comparing with other tools indicates that scHyper exhibits superior performance and functionality. Experimental results on the human tumor microenvironment and immune cells demonstrate that scHyper offers reliable and unique capabilities for analyzing intercellular communication networks. Therefore, we introduced an effective strategy that can build high-order interaction patterns, surpassing the limitations of most methods that can only handle low-order interactions, thus more accurately interpreting the complexity of intercellular communications.

HNCGAT: a method for predicting plant metabolite-protein interaction using heterogeneous neighbor contrastive graph attention network.

The prediction of metabolite-protein interactions (MPIs) plays an important role in plant basic life functions. Compared with the traditional experimental methods and the high-throughput genomics methods using statistical correlation, applying heterogeneous graph neural networks to the prediction of MPIs in plants can reduce the cost of manpower, resources, and time. However, to the best of our knowledge, applying heterogeneous graph neural networks to the prediction of MPIs in plants still remains under-explored. In this work, we propose a novel model named heterogeneous neighbor contrastive graph attention network (HNCGAT), for the prediction of MPIs in Arabidopsis. The HNCGAT employs the type-specific attention-based neighborhood aggregation mechanism to learn node embeddings of proteins, metabolites, and functional-annotations, and designs a novel heterogeneous neighbor contrastive learning framework to preserve heterogeneous network topological structures. Extensive experimental results and ablation study demonstrate the effectiveness of the HNCGAT model for MPI prediction. In addition, a case study on our MPI prediction results supports that the HNCGAT model can effectively predict the potential MPIs in plant.

Meta-learning on dynamic node clustering knowledge graph for cold-start recommendation

Meta-learning has been introduced in the recommendation domain, and a possible direction to extend graph-based meta-learning is how to exploit higher-order information between nodes. The current studies primarily rely on pre-embedding methods to utilize user attributes, inadvertently overlooking the crucial co-occurrence patterns among users’ features. This oversight hinders the discovery of implicit connections between users. Furthermore, in scenarios of infrequent interactions, the reliance on initial edge information constrains connections between nodes in the same category. This constraint compromises the update capability of node embeddings, leading to reduced recommendation accuracy. In this paper, a meta-learning recommendation method with a dynamic node clustering knowledge graph(DCKG) is proposed. By introducing user attribute nodes and a triple-gated attention networks, this approach facilitates connections between users based on attribute relationships and enables the balancing of information across various pathways. Additionally, the integration of a dynamic aggregation module frees nodes of the same category from initial connection constraints, fostering more direct connections and further augmenting the propagation capability of higher-order information. The experimental results show that DCKG achieves the excellent results in the cold-start recommendation. DCKG can extend the use of higher-order features in the graph by direct connection.

Learning from Feature and Global Topologies: Adaptive Multi-View Parallel Graph Contrastive Learning

To address the limitations of existing graph contrastive learning methods, which fail to adaptively integrate feature and topological information and struggle to efficiently capture multi-hop information, we propose an adaptive multi-view parallel graph contrastive learning framework (AMPGCL). It is an unsupervised graph representation learning method designed to generate task-agnostic node embeddings. AMPGCL constructs and encodes feature and topological views to mine feature and global topological information. To encode global topological information, we introduce an H-Transformer to decouple multi-hop neighbor aggregations, capturing global topology from node subgraphs. AMPGCL learns embedding consistency among feature, topology, and original graph encodings through a multi-view contrastive loss, generating semantically rich embeddings while avoiding information redundancy. Experiments on nine real datasets demonstrate that AMPGCL consistently outperforms thirteen state-of-the-art graph representation learning models in classification accuracy, whether in homophilous or non-homophilous graphs.

Self-supervised representations and node embedding graph neural networks for accurate and multi-scale analysis of materials

Supervised machine learning algorithms, such as graph neural networks (GNN), have successfully predicted material properties. However, the superior performance of GNN usually relies on end-to-end learning on large material datasets, which may lose the physical insight of multi-scale information about materials. And the process of labeling data consumes many resources and inevitably introduces errors, which constrains the accuracy of prediction. We propose to train the GNN model by self-supervised learning on the node and edge information of the crystal graph. Compared with the popular manually constructed material descriptors, the self-supervised atomic representation can reach better prediction performance on material properties. Furthermore, it may provide physical insights by tuning the range information. Applying the self-supervised atomic representation on the magnetic moment datasets, we show how they can extract rules and information from the magnetic materials. To incorporate rich physical information into the GNN model, we develop the node embedding graph neural networks (NEGNN) framework and show significant improvements in the prediction performance. The self-supervised material representation and the NEGNN framework may investigate in-depth information from materials and can be applied to small datasets with increased prediction accuracy.

Unifying multimodal interactions for rumor diffusion prediction with global hypergraph modeling

A central issue in rumor surveillance and management is decoding the complex dynamics of rumor propagation, with an emphasis on predicting diffusion cascades. Recent studies focus on node embedding or the sequence of dissemination in a rumor cascade based on user social interaction, while neglecting interactions between users and rumors, as well as between various rumors. Online rumor diffusion is a complex system that encompasses two fundamental components: rumors and users. A comprehensive understanding of this system’s dynamics requires a global perspective. Consequently, it is necessary to develop models that capture the inherent multimodal interactions in rumor propagation. To tackle this challenge, we propose an HG2RLink framework of rumor diffusion prediction, which unifies multimodal interactions by global encoding. Specifically, our methodology begins with the establishment of a hypergraph structure. We then refine these interactions by leveraging a hypergraph neural network that aggregates users’ preferences for rumors. Two high-order graphs are generated to capture latent spatial interactions of rumors. By employing a learning approach for multimodal interactions, each rumor diffusion sequence is modeled with a long-range vision field of users in a deep neural network. Moreover, to validate the effectiveness of our method, we introduce a new dataset available for further exploration of new methods. Finally, experimental results show that HG2RLink outperforms other methods with improvements ranging from 0.8% to 7.6% for the Hits@k metric and from 0.4% to 2.6% for the MAP@k metric across four diverse datasets.

Shapley-based graph explanation in embedding space

Graph Neural Networks (GNNs) have recently achieved remarkable success in learning graph structures and have been applied in a variety of practical applications such as medical diagnosis, drug discovery, chemical compound synthesis, and traffic forecasting. However, like other deep learning methods, GNNs are black-box networks with inexplicable results, restricting the applicability of GNNs in many important applications such as biochemistry and medicine. This paper focuses on the field of explaining graph learning methods, which is still less explored and lacks strong theoretical grounds. Based on Shapley value from game theory, we propose a novel graph explainer, named Shapley and Embedding-based Graph Explainer (SEGE), which builds a linear approximator in node embedding space to interpret the GNNs’ prediction of an input graph. Experiments on synthetic and real-world datasets demonstrate that SEGE achieves state-of-the-art performance and is significantly better than baseline models both qualitatively and quantitatively. Moreover, SEGE owns the capacity of automorphic equivalence preservation, which is not observed in existing explainers.

Unsupervised Graph Representation Learning with Inductive Shallow Node Embedding

Network science has witnessed a surge in popularity, driven by the transformative power of node representation learning for diverse applications like social network analysis and biological modeling. While shallow embedding algorithms excel at capturing network structure, they face a critical limitation—failing to generalize to unseen nodes. This paper addresses this challenge by introducing Inductive Shallow Node Embedding—as a main contribution—pioneering a novel approach that extends shallow embeddings to the realm of inductive learning. It has a novel encoder architecture that captures the local neighborhood structure of each node, enabling effective generalization to unseen nodes. In the generalization, robustness is essential to avoid degradation of performance arising from noise in the dataset. It has been theoretically proven that the covariance of the additive noise term in the proposed model is inversely proportional to the cardinality of a node’s neighbors. Another contribution is a mathematical lower bound to quantify the robustness of node embeddings, confirming its advantage over traditional shallow embedding methods, particularly in the presence of parameter noise. The proposed method demonstrably excels in dynamic networks, consistently achieving over 90% performance on previously unseen nodes compared to nodes encountered during training on various benchmarks. The empirical evaluation concludes that our method outperforms competing methods on the vast majority of datasets in both transductive and inductive tasks.

Personalizing Communication and Segmentation with Random Forest Node Embedding

Communicating effectively with customers is a challenge, especially in a context requiring long-term planning such as the pension sector. Engaging individuals to obtain information on their expected pension benefits, by personalizing the pension providers’ email communication, is a first and crucial step. We describe a machine learning approach to model email newsletters to fit individual interests. The data for the analysis is collected from newsletters sent by a pension provider of the Netherlands and is divided into two parts (N = 2,228,000 participants in total of which 465,711 participants were part of the pilot study). Our algorithm calculates node embeddings over the nodes of a random forest, which are then used as features for the machine learning task. We illustrate the algorithm’s effectiveness in classification tasks using multiple benchmark data sets, and in a data mining task where segmentation rules can be inferred from the node embeddings. The proposed model demonstrates competitive performance with respect to other approaches based on random forests, achieving the best area under the curve (AUC) in the pension data set (0.948), while also identifying customer segments that can be used by marketing departments to better target their communication towards their customers.

A complex network analysis approach to bankruptcy prediction using company relational information-based drivers

Corporate bankruptcy prediction is a long-standing topic of interest for a variety of stakeholders. Various prediction methodologies have been proposed to achieve more accurate predictions. So far, most studies have focused on making predictions using artificial intelligence models fed with accounting information only or a combination of accounting information and other category or categories of information. This study proposes new corporate governance drivers that exploit company relational information using board of directors’ networks of companies and the concept of node embeddings obtained by mapping large directors’ networks to lower-dimensional spaces. The contribution of these new drivers to the overall prediction performance is assessed using data on UK companies listed on the London Stock Exchange. The proposed two-stage methodology consists of devising new drivers in the first stage followed by risk class prediction in the second stage. We examine different complex network structures that capture the diversity of connectivity patterns observed in directors’ networks. Empirical results suggest that the proposed network-based drivers that use company relational corporate governance information significantly improve the performance of bankruptcy prediction.

Dynamic Spatio-Temporal Adaptive Graph Convolutional Recurrent Networks for Vacant Parking Space Prediction

The prediction of vacant parking spaces (VPSs) can reduce the time drivers spend searching for parking, thus alleviating traffic congestion. However, previous studies have mostly focused on modeling the temporal features of VPSs using historical data, neglecting the complex and extensive spatial characteristics of different parking lots within the transportation network. This is mainly due to the lack of direct physical connections between parking lots, making it challenging to quantify the spatio-temporal features among them. To address this issue, we propose a dynamic spatio-temporal adaptive graph convolutional recursive network (DSTAGCRN) for VPS prediction. Specifically, DSTAGCRN divides VPS data into seasonal and periodic trend components and combines daily and weekly information with node embeddings using the dynamic parameter-learning module (DPLM) to generate dynamic graphs. Then, by integrating gated recurrent units (GRUs) with the parameter-learning graph convolutional recursive module (PLGCRM) of DPLM, we infer the spatio-temporal dependencies for each time step. Furthermore, we introduce a multihead attention mechanism to effectively capture and fuse the spatio-temporal dependencies and dynamic changes in the VPS data, thereby enhancing the prediction performance. Finally, we evaluate the proposed DSTAGCRN on three real parking datasets. Extensive experiments and analyses demonstrate that the DSTAGCRN model proposed in this study not only improves the prediction accuracy but can also better extract the dynamic spatio-temporal characteristics of available parking space data in multiple parking lots.