Graph Embedding Research Articles

Relational Extraction from Biomedical Texts with Capsule Network and Hybrid Knowledge Graph Embeddings

In the expanding landscape of biomedical literature, numerous latent associations outlined in scholarly papers await discovery and integration into biomedical databases. Biomedical Natural Language Processing (NLP) research focuses on automating knowledge extraction and mining from this literature, particularly emphasizing the essential task of Relation Extraction (RE). However, existing models have limitations, mainly in their applicability to partial datasets for RE tasks. Moreover, conventional models often treat RE as a binary classification challenge, which proves suboptimal given the diverse relationships, including intricate ones like similarity and hierarchy, present in the RE dataset. These limitations are exacerbated by the models’ inability to capture word-level positional nuances and sentence-level language features. In response to these challenges, we present a novel RE model called BicapBert. This model integrates neural networks and capsule networks, enhancing them with hybrid knowledge graph embeddings to extract relevant features. BicapBert utilizes PubMedBERT and capsule networks to extract word-level positional and sentence-level language features. It further captures knowledge features from a biomedical knowledge graph, integrating them with the aforementioned linguistic features. The amalgamated information is then input into a multi-layer perceptron, culminating in results derived through a softmax classifier. Experimental evaluations on three extensive RE task datasets showcase the state-of-the-art performance of our proposed model. Additionally, we validate the model’s efficacy on three randomly selected biomedical datasets for various tasks, further affirming its superiority.

Knowledge graph embeddings based on 2d convolution and self-attention mechanisms for link prediction

Knowledge graph embeddings based on 2d convolution and self-attention mechanisms for link prediction

Interpreting single-cell and spatial omics data using deep neural network training dynamics.

Single-cell and spatial omics datasets can be organized and interpreted by annotating single cells to distinct types, states, locations or phenotypes. However, cell annotations are inherently ambiguous, as discrete labels with subjective interpretations are assigned to heterogeneous cell populations on the basis of noisy, sparse and high-dimensional data. Here we developed Annotatability, a framework for identifying annotation mismatches and characterizing biological data structure by monitoring the dynamics and difficulty of training a deep neural network over such annotated data. Following this, we developed a signal-aware graph embedding method that enables downstream analysis of biological signals. This embedding captures cellular communities associated with target signals. Using Annotatability, we address key challenges in the interpretation of genomic data, demonstrated over eight single-cell RNA sequencing and spatial omics datasets, including identifying erroneous annotations and intermediate cell states, delineating developmental or disease trajectories, and capturing cellular heterogeneity. These results underscore the broad applicability of annotation-trainability analysis via Annotatability for unraveling cellular diversity and interpreting collective cell behaviors in health and disease.

Integrability of generalised skew-symmetric replicator equations via graph embeddings

Abstract It is known that there is a one-to-one mapping between oriented directed graphs and zero-sum replicator dynamics (Lotka-Volterra equations) and that furthermore these dynamics are Hamiltonian in an appropriately defined nonlinear Poisson bracket. In this paper, we investigate the problem of determining whether these dynamics are Liouville-Arnold integrable, building on prior work graph in graph decloning by Evripidou et al. [J. Phys. A., 55:325201, 2022] and graph embedding by Paik and Griffin [Phys. Rev. E. 107(5): L052202, 2024]. Using the embedding procedure from Paik and Griffin, we show (with certain caveats) that when a graph producing integrable dynamics is embedded in another graph producing integrable dynamics, the resulting graph structure also produces integrable dynamics. We also construct a new family of graph structures that produces integrable dynamics that does not arise either from embeddings or decloning. We use these results, along with numerical methods, to classify the dynamics generated by almost all oriented directed graphs on six vertices, with three hold-out graphs that generate integrable dynamics and are not part of a natural taxonomy arising from known families and graph operations. These hold-out graphs suggest more structure is available to be found. Moreover, the work suggests that oriented directed graphs leading to integrable dynamics may be classifiable in an analogous way to the classification of finite simple groups, creating the possibility that there is a deep connection between integrable dynamics and combinatorial structures in graphs.

FedGE: Break the Scalability Limitation of Graph Neural Network With Federated Graph Embedding

Neighborhood aggregation algorithms, represented as graph convolutional networks, have attained non-negligible success in numerous topological structure-based scenarios with the assumption that the topological structure of the given graph is pre-defined and relatively small. However, a real-world graph generally is a super large graph consisting of many small graphs that are interconnected and overlapping. Which makes graph embedding in real-life industries, by nature, fall into the federated learning scheme. While current graph-based algorithms are only able to capture the individual topology of each natural graph, learning the complete structural information of the merged large graph remains challenging due to the unsustainable computational cost of graph convolutional operations. We propose a tailored federated graph embedding framework to learn the intact structural information of the numerous inherently linked small-scale graphs and the embedding of each node. We leverage graphs with around two and a half million nodes to validate the effectiveness and the correctness of the proposed framework.

Heterogeneous Graph Embedding with Dual Edge Differentiation

Recently, heterogeneous graphs have attracted widespread attention as a powerful and practical superclass of traditional homogeneous graphs, which reflect the multi-type node entities and edge relations in the real world. Most existing methods adopt meta-path construction as the mainstream to learn long-range heterogeneous semantic messages between nodes. However, such schema constructs the node-wise correlation by connecting nodes via pre-computed fixed paths, which neglects the diversities of meta-paths on the path type and path range. In this paper, we propose a meta-path-based semantic embedding schema, which is called Heterogeneous Graph Embedding with Dual Edge Differentiation (HGE-DED) to adequately construct flexible meta-path combinations thus learning the rich and discriminative semantic of target nodes. Concretely, HGE-DED devises a Multi-Type and multi-Range Meta-Path Construction (MTR-MP Construction), which covers the comprehensive exploration of meta-path combinations from path type and path range, expressing the diversity of edges at more fine-grained scales. Moreover, HGE-DED designs the semantics and meta-path joint guidance, constructing a hierarchical short- and long-range relation adjustment, which constrains the path learning as well as minimizes the impact of edge heterophily on heterogeneous graphs. Experimental results on four benchmark datasets demonstrate the effectiveness of HGE-DED compared with state-of-the-art methods.

Improving cancer driver genes identifying based on graph embedding hypergraph and hierarchical synergy dominance model

Improving cancer driver genes identifying based on graph embedding hypergraph and hierarchical synergy dominance model

Efficient Unsupervised Graph Embedding With Attributed Graph Reduction and Dual-Level Loss

Efficient Unsupervised Graph Embedding With Attributed Graph Reduction and Dual-Level Loss

Graph-DTI: A New Model for Drug-target Interaction Prediction Based on Heterogenous Network Graph Embedding.

In this study, we aimed to develop a new end-to-end learning model called Graph-Drug-Target Interaction (DTI), which integrates various types of information in the heterogeneous network data, and to explore automatic learning of the topology-maintaining representations of drugs and targets, thereby effectively contributing to the prediction of DTI. Precise predictions of DTI can guide drug discovery and development. Most machine learning algorithms integrate multiple data sources and combine them with common embedding methods. However, the relationship between the drugs and target proteins is not well reported. Although some existing studies have used heterogeneous network graphs for DTI prediction, there are many limitations in the neighborhood information between the nodes in the heterogeneous network graphs. We studied the drug-drug interaction (DDI) and DTI from DrugBank Version 3.0, protein-protein interaction (PPI) from the human protein reference database Release 9, drug structure similarity from Morgan fingerprints of radius 2 and calculated by RDKit, and protein sequence similarity from Smith-Waterman score. Our study consists of three major components. First, various drugs and target proteins were integrated, and a heterogeneous network was established based on a series of data sets. Second, the graph neural networks-inspired graph auto-encoding method was used to extract high-order structural information from the heterogeneous networks, thereby revealing the description of nodes (drugs and proteins) and their topological neighbors. Finally, potential DTI prediction was made, and the obtained samples were sent to the classifier for secondary classification. The performance of Graph-DTI and all baseline methods was evaluated using the sums of the area under the precision-recall curve (AUPR) and the area under the receiver operating characteristic curve (AUC). The results indicated that Graph-DTI outperformed the baseline methods in both performance results. Compared with other baseline DTI prediction methods, the results showed that Graph-DTI had better prediction performance. Additionally, in this study, we effectively classified drugs corresponding to different targets and vice versa. The above findings showed that Graph-DTI provided a powerful tool for drug research, development, and repositioning. Graph- DTI can serve as a drug development and repositioning tool more effectively than previous studies that did not use heterogeneous network graph embedding.

Wasserstein Discriminant Dictionary Learning for Graph Representation.

Mining discriminative graph topological information plays an important role in promoting graph representation ability. However, it suffers from two main issues: (1) the difficulty/complexity of computing global inter-class/intra-class scatters, commonly related to mean and covariance of graph samples, for discriminant learning; (2) the huge complexity and variety of graph topological structure that is rather challenging to robustly characterize. In this paper, we propose the Wasserstein Discriminant Dictionary Learning (WDDL) framework to achieve discriminant learning on graphs with robust graph topology modeling, and hence facilitate graph-based pattern analysis tasks. Considering the difficulty of calculating global inter-class/intra-class scatters, a reference set of graphs (aka graph dictionary) is first constructed by generating representative graph samples (aka graph keys) with expressive topological structure. Then, a Wasserstein Graph Representation (WGR) process is proposed to project input graphs into a succinct dictionary space through the graph dictionary lookup. To further achieve discriminant graph learning, a Wasserstein discriminant loss (WD-loss) is defined on the graph dictionary, in which the graph keys are optimizable, to make the intra-class keys more compact and inter-class keys more dispersed. Hence, the calculation of global Wasserstein metric (W-metric) centers can be bypassed. For sophisticated topology mining in the WGR process, a joint-Wasserstein graph embedding module is constructed to model both between-node and between-edge relationships across inputs and graph keys by encapsulating both the Wasserstein metric (between cross-graph nodes) and proposed novel Kron-Gromov-Wasserstein (KGW) metric (between cross-graph adjacencies). Specifically, the KGW-metric comprehensively characterizes the cross-graph connection patterns with the Kronecker operation, then adaptively captures those salient patterns through connection pooling. To evaluate the proposed framework, we study two graph-based pattern analysis problems, i.e. graph classification and cross-modal retrieval, with the graph dictionary flexibly adjusted to cater to these two tasks. Extensive experiments are conducted to comprehensively compare with existing advanced methods, as well as dissect the critical component of our proposed architecture. The experimental results validate the effectiveness of the WDDL framework.

RAG-based explainable prediction of road users behaviors for automated driving using knowledge graphs and large language models

The prediction of road user behaviors in the context of autonomous driving has attracted considerable attention from the scientific community in recent years. Most works focus on predicting behaviors based on kinematic information alone, a simplification of reality since road users are humans, and as such they are highly influenced by their surrounding context. In addition, a large plethora of research works rely on powerful Deep Learning techniques, which exhibit high-performance metrics in prediction tasks but may lack the ability to fully understand and exploit the contextual semantic information contained in the road scene, not to mention their inability to provide explainable predictions that can be understood by humans. In this work, we propose an explainable road users’ behavior prediction system that integrates the reasoning abilities of Knowledge Graphs (KG) and the expressiveness capabilities of Large Language Models (LLM) by using Retrieval Augmented Generation (RAG) techniques. For that purpose, Knowledge Graph Embeddings (KGE) and Bayesian inference are combined to allow the deployment of a fully inductive reasoning system that enables the issuing of predictions that rely on legacy information contained in the graph, as well as on current evidence gathered in real-time by onboard sensors. Two use cases have been implemented following the proposed approach: 1) Prediction of pedestrians’ crossing actions; and 2) Prediction of lane change maneuvers. In both cases, the performance attained exceeds the current state-of-the-art in terms of anticipation and F1 score, showing a promising avenue for future research in this field.

Exploratory analysis of metabolic changes using mass spectrometry data and graph embeddings

Mass spectrometry (MS)-based metabolomics analysis is a powerful tool, but it comes with its own set of challenges. The MS workflow involves multiple steps before its interpretation in what is denominate data mining. Data mining consists of a two-step process. First, the MS data is ordered, arranged, and presented for filtering before being analyzed. Second, the filtered and reduced data are analyzed using statistics to remove further variability. This holds true particularly for MS-based untargeted metabolomics studies, which focused on understanding fold changes in metabolic networks. Since the task of filtering and identifying changes from a large dataset is challenging, automated techniques for mining untargeted MS-based metabolomic data are needed. The traditional statistics-based approach tends to overfilter raw data, which may result in the removal of relevant data and lead to the identification of fewer metabolomic changes. This limitation of the traditional approach underscores the need for a new method. In this work, we present a novel deep learning approach using node embeddings (powered by GNNs), edge embeddings, and anomaly detection algorithm to analyze the data generated by mass spectrometry (MS)-based metabolomics called GEMNA (Graph Embedding-based Metabolomics Network Analysis), for example for an untargeted volatile study on Mentos candy, the data clusters produced by GEMNA were better than the ones used traditional tools, i.e., GEMNA has \\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$silhouette\\,score = 0.409$$\\end{document}, vs. the traditional approach has \\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$silhouette\\,score = -0.004$$\\end{document}.

Accurate Spatial Heterogeneity Dissection and Gene Regulation Interpretation for Spatial Transcriptomics using Dual Graph Contrastive Learning.

Recent advances in spatial transcriptomics have enabled simultaneous preservation of high-throughput gene expression profiles and the spatial context, enabling high-resolution exploration of distinct regional characterization in tissue. To effectively understand the underlying biological mechanisms within tissue microenvironments, there is a requisite for methods that can accurately capture external spatial heterogeneity and interpret internal gene regulation from spatial transcriptomics data. However, current methods for region identification often lack the simultaneous characterizing of spatial structure and gene regulation, thereby limiting the ability of spatial dissection and gene interpretation. Here, stDCL is developed, a dual graph contrastive learning method to identify spatial domains and interpret gene regulation in spatial transcriptomics data. stDCL adaptively incorporates gene expression data and spatial information via a graph embedding autoencoder, thereby preserving critical information within the latent embedding representations. In addition, dual graph contrastive learning is proposed to train the model, ensuring that the latent embedding representation closely resembles the actual spatial distribution and exhibits cluster similarity. Benchmarking stDCL against other state-of-the-art clustering methods using complex cortex datasets demonstrates its superior accuracy and effectiveness in identifying spatial domains. Ouranalysis of the imputation matrices generated by stDCL reveals its capability to reconstruct spatial hierarchical structures and refine differential expression assessment. Furthermore, it is demonstrated that the versatility of stDCL in interpretability of gene regulation, spatial heterogeneity at high resolution, and embryonic developmental patterns. In addition, it is also showed that stDCL can successfully annotate disease-associated astrocyte subtypes in Alzheimer's disease and unravel multiple relevant pathways and regulatorymechanisms.

A data-centric framework of improving graph neural networks for knowledge graph embedding

A data-centric framework of improving graph neural networks for knowledge graph embedding

Multi-angle information aggregation for inductive temporal graph embedding.

Graph embedding has gained significant popularity due to its ability to represent large-scale graph data by mapping nodes to a low-dimensional space. However, most of the existing research in this field has focused on transductive learning, where fixed node embeddings are generated by training the entire graph. This approach is not well-suited for temporal graphs that undergo continuous changes with the addition of new nodes and interactions. To address this limitation, we propose an inductive temporal graph embedding method called MIAN (Multi-angle Information Aggregation Network). The key focus of MIAN is to design an aggregation function that combines multi-angle information for generating node embeddings. Specifically, we divide the information into different angles, including neighborhood, temporal, and environment. Each angle of information is modeled and mined independently, and then fed into an improved gated recuttent unit (GRU) module to effectively combine them. To assess the performance of MIAN, we conduct extensive experiments on various real-world datasets and compare its results with several state-of-the-art baseline methods across diverse tasks. The experimental findings demonstrate that MIAN outperforms these methods.

Directed Knowledge Graph Embedding Using a Hybrid Architecture of Spatial and Spectral GNNs

Knowledge graph embedding has been identified as an effective method for node-level classification tasks in directed graphs, the objective of which is to ensure that nodes of different categories are embedded as far apart as possible in the feature space. The directed graph is a general representation of unstructured knowledge graphs. However, existing methods lack the ability to simultaneously approximate high-order filters and globally pay attention to the task-related connectivity between distant nodes for directed graphs. To address this limitation, a directed spectral graph transformer (DSGT), a hybrid architecture model, is constructed by integrating the graph transformer and directed spectral graph convolution networks. The graph transformer leverages multi-head attention mechanisms to capture the global connectivity of the feature graph from different perspectives in the spatial domain, which bridges the gap between frequency responses and, further, naturally couples the graph transformer and directed graph convolutional neural networks (GCNs). In addition to the inherent hard inductive bias of DSGT, we introduce directed node positional and structure-aware edge embedding to provide topological prior knowledge. Extensive experiments demonstrate that the DSGT exhibits state-of-the-art (SOTA) or competitive node-level representation capabilities across datasets of varying attributes and scales. Furthermore, the experimental results indicate that the homophily and degree of correlation of the nodes significantly influence the classification performance of the model. This finding opens significant avenues for future research.

Fusing multiplex heterogeneous networks using graph attention-aware fusion networks

Graph Neural Networks (GNN) emerged as a deep learning framework to generate node and graph embeddings for downstream machine learning tasks. Popular GNN-based architectures operate on networks of single node and edge type. However, a large number of real-world networks include multiple types of nodes and edges. Enabling these architectures to work on networks with multiple node and edge types brings additional challenges due to the heterogeneity of the networks and the multiplicity of the existing associations. In this study, we present a framework, named GRAF (Graph Attention-aware Fusion Networks), to convert multiplex heterogeneous networks to homogeneous networks to make them more suitable for graph representation learning. Using attention-based neighborhood aggregation, GRAF learns the importance of each neighbor per node (called node-level attention) followed by the importance of each network layer (called network layer-level attention). Then, GRAF processes a network fusion step weighing each edge according to the learned attentions. After an edge elimination step based on edge weights, GRAF utilizes Graph Convolutional Networks (GCN) on the fused network and incorporates node features on graph-structured data for a node classification or a similar downstream task. To demonstrate GRAF’s generalizability, we applied it to four datasets from different domains and observed that GRAF outperformed or was on par with the baselines and state-of-the-art (SOTA) methods. We were able to interpret GRAF’s findings utilizing the attention weights. Source code for GRAF is publicly available at https://github.com/bozdaglab/GRAF.

MoAGL-SA: a multi-omics adaptive integration method with graph learning and self attention for cancer subtype classification

BackgroundThe integration of multi-omics data through deep learning has greatly improved cancer subtype classification, particularly in feature learning and multi-omics data integration. However, key challenges remain in embedding sample structure information into the feature space and designing flexible integration strategies.ResultsWe propose MoAGL-SA, an adaptive multi-omics integration method based on graph learning and self-attention, to address these challenges. First, patient relationship graphs are generated from each omics dataset using graph learning. Next, three-layer graph convolutional networks are employed to extract omic-specific graph embeddings. Self-attention is then used to focus on the most relevant omics, adaptively assigning weights to different graph embeddings for multi-omics integration. Finally, cancer subtypes are classified using a softmax classifier.ConclusionsExperimental results show that MoAGL-SA outperforms several popular algorithms on datasets for breast invasive carcinoma, kidney renal papillary cell carcinoma, and kidney renal clear cell carcinoma. Additionally, MoAGL-SA successfully identifies key biomarkers for breast invasive carcinoma.

Advances in the application of network analysis methods in traditional Chinese medicine research

ObjectiveThis review aims at evaluating the role and potential applications of network analysis methods in the medicinal substances of traditional Chinese medicine (TCM), theories of TCM compatibility, properties of herbs, and TCM syndromes. MethodsLiterature was retrieved from databases, such as CNKI, PubMed, and Web of Science, using keywords, including "network analysis," "network biology," "network pharmacology," and "network medicine." The extracted literature included the biological network construction (including ingredient-target and target-disease relations), analysis of network topology characteristics (including node degree, clustering coefficient, and path length), network modularization analysis, functional annotation and so on. These studies were categorized and organized based on their research methods, application domains, and other relevant characteristics. ResultsNetwork analysis algorithms, such as network distance, random walk, matrix factorization, graph embedding, and graph neural networks, are widely applied in fields related to the properties, compatibility, and mechanisms of TCM. They effectively reflect the interactive relations within the complex systems of TCM and elucidate and clarify theories, such as the effective substances, the principles of TCM compatibility, the TCM syndromes, and the properties of TCM. ConclusionThe network analysis method is a powerful mathematical and computational tool that reveals the structure, dynamics, and functions of complex systems by analyzing the elements and their relations. This approach has effectively promoted the modernization of TCM, providing essential theoretical and practical tools for personalized treatment and scientific research on TCM. It also offers a significant methodological framework for the modernization and internationalization of TCM.

Customer Segmentation and Need Analysis Based on Sentiment Network of Online Reviewers and Graph Embedding

Customer Segmentation and Need Analysis Based on Sentiment Network of Online Reviewers and Graph Embedding