Graph Representation Learning Framework Research Articles

Learning Graph Representations Through Learning and Propagating Edge Features.

Graph convolutional networks have achieved considerable success in various graph domain tasks. Recently, numerous types of graph convolutional networks have been developed. A typical rule for learning a node's feature in these graph convolutional networks is to aggregate node features from the node's local neighborhood. However, in these models, the interrelation information between adjacent nodes is not well-considered. This information could be helpful to learn improved node embeddings. In this article, we present a graph representation learning framework that generates node embeddings through learning and propagating edge features. Instead of aggregating node features from a local neighborhood, we learn a feature for each edge and update a node's representation by aggregating local edge features. The edge feature is learned from the concatenation of the edge's starting node feature, the input edge feature, and the edge's end node feature. Unlike node feature propagation-based graph networks, our model propagates different features from a node to its neighbors. In addition, we learn an attention vector for each edge in aggregation, enabling the model to focus on important information in each feature dimension. By learning and aggregating edge features, the interrelation between a node and its neighboring nodes is integrated in the aggregated feature, which helps learn improved node embeddings in graph representation learning. Our model is evaluated on graph classification, node classification, graph regression, and multitask binary graph classification on eight popular datasets. The experimental results demonstrate that our model achieves improved performance compared with a wide variety of baseline models.

Self-attention empowered graph convolutional network for structure learning and node embedding

In representation learning on graph-structured data, many popular graph neural networks (GNNs) fail to capture long-range dependencies, leading to performance degradation. Furthermore, this weakness is magnified when the concerned graph is characterized by heterophily (low homophily). To solve this issue, this paper proposes a novel graph representation learning framework called the graph convolutional network with self-attention (GCN-SA). The proposed scheme exhibits an exceptional generalization capability in node-level representation learning. The proposed GCN-SA contains two enhancements corresponding to edges and node features. For edges, we utilize a self-attention mechanism to design a stable and effective graph-structure-learning module that can capture the internal correlation between any pair of nodes. This graph-structure-learning module can identify reliable neighbors for each node from the entire graph. Regarding the node features, we modify the transformer block to enable it to assist the GCN in integrating valuable information from the entire graph. These two enhancements work in distinct ways to help our GCN-SA capture long-range dependencies, enabling it to perform representation learning on graphs with varying levels of homophily. The experimental results on real-world benchmark datasets demonstrate the effectiveness of the proposed GCN-SA. Compared to other outstanding GNN counterparts, the proposed GCN-SA is competitive. The source code is available at https://github.com/mengyingjiang/GCN-SA.

Dual-Channel Learning Framework for Drug-Drug Interaction Prediction via Relation-Aware Heterogeneous Graph Transformer

Identifying novel drug-drug interactions (DDIs) is a crucial task in pharmacology, as the interference between pharmacological substances can pose serious medical risks. In recent years, several network-based techniques have emerged for predicting DDIs. However, they primarily focus on local structures within DDI-related networks, often overlooking the significance of indirect connections between pairwise drug nodes from a global perspective. Additionally, effectively handling heterogeneous information present in both biomedical knowledge graphs and drug molecular graphs remains a challenge for improved performance of DDI prediction. To address these limitations, we propose a Transformer-based relatIon-aware Graph rEpresentation leaRning framework (TIGER) for DDI prediction. TIGER leverages the Transformer architecture to effectively exploit the structure of heterogeneous graph, which allows it direct learning of long dependencies and high-order structures. Furthermore, TIGER incorporates a relation-aware self-attention mechanism, capturing a diverse range of semantic relations that exist between pairs of nodes in heterogeneous graph. In addition to these advancements, TIGER enhances predictive accuracy by modeling DDI prediction task using a dual-channel network, where drug molecular graph and biomedical knowledge graph are fed into two respective channels. By incorporating embeddings obtained at graph and node levels, TIGER can benefit from structural properties of drugs as well as rich contextual information provided by biomedical knowledge graph. Extensive experiments conducted on three real-world datasets demonstrate the effectiveness of TIGER in DDI prediction. Furthermore, case studies highlight its ability to provide a deeper understanding of underlying mechanisms of DDIs.

Graph Representation Learning Based on Specific Subgraphs for Biomedical Interaction Prediction.

Discovering the novel associations of biomedical entities is of great significance and can facilitate not only the identification of network biomarkers of disease but also the search for putative drug targets. Graph representation learning(GRL) has incredible potential to efficiently predict the interactions from biomedical networks by modeling the robust representation for each node. However, the current GRL-based methods learn the representation of nodes by aggregating the features of their neighbors with equal weights. Furthermore, they also fail to identify which features of higher-order neighbors are integrated into the representation of the central node. In this work, we propose a novel graph representation learning framework: a multi-order graph neural network based on reconstructed specific subgraphs(MGRS) for biomedical interaction prediction. In the MGRS, we apply the multi-order graph aggregation module(MOGA) to learn the wide-view representation by integrating the multi-hop neighbor features. Besides, we propose a subgraph selection module(SGSM) to reconstruct the specific subgraph with adaptive edge weights for each node. SGSM can clearly explore the dependency of the node representation on the neighbor features and learn the subgraph-based representation based on the reconstructed weighted subgraphs. Extensive experimental results on four public biomedical networks demonstrate that the MGRS performs better and is more robust than the latest baselines.

HKTSG: A Hierarchical Knowledge-guided Traffic Scene Graph Representation Learning Framework for Intelligent Vehicles

HKTSG: A Hierarchical Knowledge-guided Traffic Scene Graph Representation Learning Framework for Intelligent Vehicles

CLEAR: Cluster-Enhanced Contrast for Self-Supervised Graph Representation Learning.

This article studies self-supervised graph representation learning, which is critical to various tasks, such as protein property prediction. Existing methods typically aggregate representations of each individual node as graph representations, but fail to comprehensively explore local substructures (i.e., motifs and subgraphs), which also play important roles in many graph mining tasks. In this article, we propose a self-supervised graph representation learning framework named cluster-enhanced Contrast (CLEAR) that models the structural semantics of a graph from graph-level and substructure-level granularities, i.e., global semantics and local semantics, respectively. Specifically, we use graph-level augmentation strategies followed by a graph neural network-based encoder to explore global semantics. As for local semantics, we first use graph clustering techniques to partition each whole graph into several subgraphs while preserving as much semantic information as possible. We further employ a self-attention interaction module to aggregate the semantics of all subgraphs into a local-view graph representation. Moreover, we integrate both global semantics and local semantics into a multiview graph contrastive learning framework, enhancing the semantic-discriminative ability of graph representations. Extensive experiments on various real-world benchmarks demonstrate the efficacy of the proposed over current graph self-supervised representation learning approaches on both graph classification and transfer learning tasks.

Predicting circRNA-drug resistance associations based on a multimodal graph representation learning framework.

Circular RNA (circRNA) is a class of noncoding RNA that is highly conserved and exhibit exceptional stability. Due to its function as a microRNA sponge, circRNA has gained significant attention as an essential biomarker and potential drug target in the pathogenesis of several cancers. Although many circRNAs have been identified to play a role in cancer resistance, traditional methods are time-consuming and expensive. In this context, computational methods offer a promising way to facilitate the discovery process. However, most existing prediction models focus on the association between circRNAs and drug resistance, without considering the corresponding disease-related information in the circRNA-drug resistance association. Incorporating disease-related information into the prediction of circRNA-drug resistance associations could potentially improve the efficiency and speed of discovering and developing circRNA-targeting drugs. We propose a computational framework, named GraphCDD, for predicting the association between circRNA and drug resistance. Our model utilizes data from three sources, namely circRNA, disease, and drug, to construct three similarity networks that represent the features of circRNA, disease, and drug, respectively. We utilize a multimodal graph neural network to acquire efficient representations of circRNAs, diseases, and drugs by integrating various types of information, and establish a predictive model. The experimental results have validated the effectiveness of our model and provided a promising method in predicting potential associations between circRNA and drug resistance. The source code and dataset of GraphCDD can be found at https://github.com/Ziqiang-Liu/GraphCDD.

Speos: an ensemble graph representation learning framework to predict core gene candidates for complex diseases

Understanding phenotype-to-genotype relationships is a grand challenge of 21st century biology with translational implications. The recently proposed “omnigenic” model postulates that effects of genetic variation on traits are mediated by core-genes and -proteins whose activities mechanistically influence the phenotype, whereas peripheral genes encode a regulatory network that indirectly affects phenotypes via core gene products. Here, we develop a positive-unlabeled graph representation-learning ensemble-approach based on a nested cross-validation to predict core-like genes for diverse diseases using Mendelian disorder genes for training. Employing mouse knockout phenotypes for external validations, we demonstrate that core-like genes display several key properties of core genes: Mouse knockouts of genes corresponding to our most confident predictions give rise to relevant mouse phenotypes at rates on par with the Mendelian disorder genes, and all candidates exhibit core gene properties like transcriptional deregulation in disease and loss-of-function intolerance. Moreover, as predicted for core genes, our candidates are enriched for drug targets and druggable proteins. In contrast to Mendelian disorder genes the new core-like genes are enriched for druggable yet untargeted gene products, which are therefore attractive targets for drug development. Interpretation of the underlying deep learning model suggests plausible explanations for our core gene predictions in form of molecular mechanisms and physical interactions. Our results demonstrate the potential of graph representation learning for the interpretation of biological complexity and pave the way for studying core gene properties and future drug development.

IHerd: an integrative hierarchical graph representation learning framework to quantify network changes and prioritize risk genes in disease.

Different genes form complex networks within cells to carry out critical cellular functions, while network alterations in this process can potentially introduce downstream transcriptome perturbations and phenotypic variations. Therefore, developing efficient and interpretable methods to quantify network changes and pinpoint driver genes across conditions is crucial. We propose a hierarchical graph representation learning method, called iHerd. Given a set of networks, iHerd first hierarchically generates a series of coarsened sub-graphs in a data-driven manner, representing network modules at different resolutions (e.g., the level of signaling pathways). Then, it sequentially learns low-dimensional node representations at all hierarchical levels via efficient graph embedding. Lastly, iHerd projects separate gene embeddings onto the same latent space in its graph alignment module to calculate a rewiring index for driver gene prioritization. To demonstrate its effectiveness, we applied iHerd on a tumor-to-normal GRN rewiring analysis and cell-type-specific GCN analysis using single-cell multiome data of the brain. We showed that iHerd can effectively pinpoint novel and well-known risk genes in different diseases. Distinct from existing models, iHerd's graph coarsening for hierarchical learning allows us to successfully classify network driver genes into early and late divergent genes (EDGs and LDGs), emphasizing genes with extensive network changes across and within signaling pathway levels. This unique approach for driver gene classification can provide us with deeper molecular insights. The code is freely available at https://github.com/aicb-ZhangLabs/iHerd. All other relevant data are within the manuscript and supporting information files.

Self-Supervised Dynamic Graph Representation Learning via Temporal Subgraph Contrast

Self-supervised learning on graphs has recently drawn a lot of attention due to its independence from labels and its robustness in representation. Current studies on this topic mainly use static information such as graph structures but cannot well capture dynamic information such as timestamps of edges. Realistic graphs are often dynamic, which means the interaction between nodes occurs at a specific time. This article proposes a self-supervised dynamic graph representation learning framework DySubC, which defines a temporal subgraph contrastive learning task to simultaneously learn the structural and evolutional features of a dynamic graph. Specifically, a novel temporal subgraph sampling strategy is firstly proposed, which takes each node of the dynamic graph as the central node and uses both neighborhood structures and edge timestamps to sample the corresponding temporal subgraph. The subgraph representation function is then designed according to the influence of neighborhood nodes on the central node after encoding the nodes in each subgraph. Finally, the structural and temporal contrastive loss are defined to maximize the mutual information between node representation and temporal subgraph representation. Experiments on five real-world datasets demonstrate that (1) DySubC performs better than the related baselines including two graph contrastive learning models and five dynamic graph representation learning models, especially in the link prediction task, and (2) the use of temporal information cannot only sample more effective subgraphs, but also learn better representation by temporal contrastive loss.

A Graph Representation Learning Framework Predicting Potential Multivariate Interactions

Link prediction is a widely adopted method for extracting valuable data insights from graphs, primarily aimed at predicting interactions between two nodes. However, there are not only pairwise interactions but also multivariate interactions in real life. For example, reactions between multiple proteins, multiple compounds, and multiple metabolites cannot be mined effectively using link prediction. A hypergraph is a higher-order network composed of nodes and hyperedges, where hyperedges can be composed of multiple nodes, and can be used to depict multivariate interactions. The interactions between multiple nodes can be predicted by hyperlink prediction methods. Since hyperlink prediction requires predicting the interactions between multiple nodes, it makes the study of hyperlink prediction much more complicated than that of other complex networks, thus resulting in relatively limited attention being devoted to this field. The existing hyperlink prediction can only predict potential hyperlinks in uniform hypergraphs, or need to predict hyperlinks based on the candidate hyperlink sets, or only study hyperlink prediction for undirected hypergraphs. Therefore, a hyperlink prediction framework for predicting multivariate interactions based on graph representation learning is proposed to solve the above problems, and then the framework is extended to directed hyperlink prediction (e.g., directed metabolic reaction networks). Furthermore, any size of hyperedges can be predicted by the proposed hyperlink prediction algorithm framework, whose performance is not affected by the number of nodes or the number of hyperedges. Finally, the proposed framework is applied to both the biological metabolic reaction network and the organic chemical reaction network, and experimental analysis has demonstrated that the hyperlinks can be predicted efficiently by the proposed hyperlink prediction framework with relatively low time complexity, and the prediction performance has been improved by up to 40% compared with the baselines.

Property graph representation learning for node classification

Graph representation learning (graph embedding) has led to breakthrough results in various machine learning graph-based applications such as node classification, link prediction and recommendation. Many real-world graphs can be characterized as the property graphs, because besides the structure information, there exists rich property information related to each node in the graphs. Many existing graph representation learning methods—e.g. random walk-based methods like DeepWalk and Node2vec, focus only on the structure of graph for learning the node embedding. Although graph representation learning based on neural networks (e.g. typical GNN methods such as GraphSAGE) uses the property of nodes as the initial features of nodes and then aggregates feature information of the neighbours, their limitation is that the neighbourhood of a node is considered to be uniform—i.e. there is no way to differentiate among neighbours of a node when learning a node embedding. Additionally, their definition of neighbourhood is local, i.e. only nodes connected to the current node are considered as neighbours. Hence, those methods fail to capture implicit/latent relationships among nodes, which are implicit in the given structure. In this study, our aim is to improve the performance of graph representation learning methods on property graphs. We present a new framework called Enhanced Property Graph Embedding (EPGE)—a graph representation learning framework to address above-mentioned limitations. Our proposed framework relies on the notion of latent neighbourhood, as well as systematic sampling of neighbouring nodes to obtain better representation of the nodes. The experimental results on five publicly available graph datasets demonstrate that EPGE outperforms state-of-the-art baselines for the task of node classification. We further evaluate the superiority of our proposed formulation by defining a novel quantitative metric to measure the usefulness of the sampled neighbourhood in the graph.

HimGNN: a novel hierarchical molecular graph representation learning framework for property prediction.

Accurate prediction of molecular properties is an important topic in drug discovery. Recent works have developed various representation schemes for molecular structures to capture different chemical information in molecules. The atom and motif can be viewed as hierarchical molecular structures that are widely used for learning molecular representations to predict chemical properties. Previous works have attempted to exploit both atom and motif to address the problem of information loss in single representation learning for various tasks. To further fuse such hierarchical information, the correspondence between learned chemical features from different molecular structures should be considered. Herein, we propose a novel framework for molecular property prediction, called hierarchical molecular graph neural networks (HimGNN). HimGNN learns hierarchical topology representations by applying graph neural networks on atom- and motif-based graphs. In order to boost the representational power of the motif feature, we design a Transformer-based local augmentation module to enrich motif features by introducing heterogeneous atom information in motif representation learning. Besides, we focus on the molecular hierarchical relationship and propose a simple yet effective rescaling module, called contextual self-rescaling, that adaptively recalibrates molecular representations by explicitly modelling interdependencies between atom and motif features. Extensive computational experiments demonstrate that HimGNN can achieve promising performances over state-of-the-art baselines on both classification and regression tasks in molecular property prediction.

A One-Size-Fits-Three Representation Learning Framework for Patient Similarity Search

AbstractPatient similarity search is an essential task in healthcare. Recent studies adopted electronic health records (EHRs) to learn patient representations for measuring the clinical similarities. These methods outperformed traditional methods, by capturing more information from various sources consisting of multi-modal EHRs, external knowledge and correlations among medical concepts. They often concerned certain type of data without taking full advantage of various information. We propose a graph representation learning framework, denoted by One-Size-Fits-Three (OSFT), that takes into account fusion-attention, neighbor-attention and global-attention from three types of information. Extensive experiments are conducted on two real datasets of MIMIC-III and MIMIC-IV, and the results verified the effectiveness and generality of our framework. When compared with baselines on patient similarity search, our framework achieved good effectiveness and comparative efficiency. The results provide new insights about whether the use of various information can better measure the patient similarity. The source codes are available at https://github.com/emmali808/ADDS/tree/master/EHRDeepHelper.

Signed Laplacian Graph Neural Networks

This paper studies learning meaningful node representations for signed graphs, where both positive and negative links exist. This problem has been widely studied by meticulously designing expressive signed graph neural networks, as well as capturing the structural information of the signed graph through traditional structure decomposition methods, e.g., spectral graph theory. In this paper, we propose a novel signed graph representation learning framework, called Signed Laplacian Graph Neural Network (SLGNN), which combines the advantages of both. Specifically, based on spectral graph theory and graph signal processing, we first design different low-pass and high-pass graph convolution filters to extract low-frequency and high-frequency information on positive and negative links, respectively, and then combine them into a unified message passing framework. To effectively model signed graphs, we further propose a self-gating mechanism to estimate the impacts of low-frequency and high-frequency information during message passing. We mathematically establish the relationship between the aggregation process in SLGNN and signed Laplacian regularization in signed graphs, and theoretically analyze the expressiveness of SLGNN. Experimental results demonstrate that SLGNN outperforms various competitive baselines and achieves state-of-the-art performance.

SimGRL: a simple self-supervised graph representation learning framework via triplets

Recently, graph contrastive learning (GCL) has achieved remarkable performance in graph representation learning. However, existing GCL methods usually follow a dual-channel encoder network (i.e., Siamese networks), which adds to the complexity of the network architecture. Additionally, these methods overly depend on varied data augmentation techniques, corrupting graph information. Furthermore, they are heavily reliant on large quantities of negative nodes for each object node, which requires tremendous memory costs. To address these issues, we propose a novel and simple graph representation learning framework, named SimGRL. Firstly, our proposed network architecture only contains one encoder based on a graph neural network instead of a dual-channel encoder, which simplifies the network architecture. Then we introduce a distributor to generate triplets to obtain the contrastive views between nodes and their neighbors, avoiding the need for data augmentations. Finally, we design a triplet loss based on adjacency information in graphs that utilizes only one negative node for each object node, reducing memory overhead significantly. Extensive experiments demonstrate that SimGRL achieves competitive performance on node classification and graph classification tasks, especially in terms of running time and memory overhead.

A social theory-enhanced graph representation learning framework for multitask prediction of drug-drug interactions.

Current machine learning-based methods have achieved inspiring predictions in the scenarios of mono-type and multi-type drug-drug interactions (DDIs), but they all ignore enhancive and depressive pharmacological changes triggered by DDIs. In addition, these pharmacological changes are asymmetric since the roles of two drugs in an interaction are different. More importantly, these pharmacological changes imply significant topological patterns among DDIs. To address the above issues, we first leverage Balance theory and Status theory in social networks to reveal the topological patterns among directed pharmacological DDIs, which are modeled as a signed and directed network. Then, we design a novel graph representation learning model named SGRL-DDI (social theory-enhanced graph representation learning for DDI) to realize the multitask prediction of DDIs. SGRL-DDI model can capture the task-joint information by integrating relation graph convolutional networks with Balance and Status patterns. Moreover, we utilize task-specific deep neural networks to perform two tasks, including the prediction of enhancive/depressive DDIs and the prediction of directed DDIs. Based on DDI entries collected from DrugBank, the superiority of our model is demonstrated by the comparison with other state-of-the-art methods. Furthermore, the ablation study verifies that Balance and Status patterns help characterize directed pharmacological DDIs, and that the joint of two tasks provides better DDI representations than individual tasks. Last, we demonstrate the practical effectiveness of our model by a version-dependent test, where 88.47 and 81.38% DDI out of newly added entries provided by the latest release of DrugBank are validated in two predicting tasks respectively.

BrainTGL: A dynamic graph representation learning model for brain network analysis

Modeling the dynamics characteristics in functional brain networks (FBNs) is important for understanding the functional mechanism of the human brain. However, the current works do not fully consider the potential complex spatial and temporal correlations in human brain. To solve this problem, we propose a temporal graph representation learning framework for brain networks (BrainTGL). The framework involves a temporal graph pooling for eliminating the noisy edges as well as data inconsistency, and a dual temporal graph learning for capturing the spatio-temporal features of the temporal graphs. The proposed method has been evaluated in both tasks of brain disease (ASD, MDD and BD) diagnosis/gender classification (classification task) and subtype identification (clustering task) on the four datasets: Human Connectome Project (HCP), Autism Brain Imaging Data Exchange (ABIDE), NMU-MDD and NMU-BD. A large improvement is achieved for the ASD diagnosis. Specifically, our model outperforms the GroupINN and ST-GCN by an average increase of 4.2% and 8.6% on accuracy, respectively, demonstrating its advantages in comparison to the state-of-the-art methods based on functional connectivity features or learned spatio-temporal features. The results demonstrate that learning the spatial–temporal brain network representation for modeling dynamics characteristics in FBNs can improve the model’s performance on both disease diagnosis and subtype identification tasks for multiple disorders. Apart from performance, the improvements of computational efficiency and convergence speed reduce training costs.

UniSKGRep: A unified representation learning framework of social network and knowledge graph

The human-oriented applications aim to exploit behaviors of people, which impose challenges on user modeling of integrating social network (SN) with knowledge graph (KG), and jointly analyzing two types of graph data. However, existing graph representation learning methods merely represent one of two graphs alone, and hence are unable to comprehensively consider features of both SN and KG with profiling the correlation between them, resulting in unsatisfied performance in downstream tasks. Considering the diverse gap of features and the difficulty of associating of the two graph data, we introduce a Unified Social Knowledge Graph Representation learning framework (UniSKGRep), with the goal to leverage the multi-view information inherent in the SN and KG for improving the downstream tasks of user modeling. To the best of our knowledge, we are the first to present a unified representation learning framework for SN and KG. Concretely, the SN and KG are organized as the Social Knowledge Graph (SKG), a unified representation of SN and KG. For the representation learning of SKG, first, two separate encoders in the Intra-graph model capture both the social-view and knowledge-view in two embedding spaces, respectively. Then the Inter-graph model is learned to associate the two separate spaces via bridging the semantics of overlapping node pairs. In addition, the overlapping node enhancement module is designed to effectively align two spaces with the consideration of a relatively small number of overlapping nodes. The two spaces are gradually unified by continuously iterating the joint training procedure. Extensive experiments on two real-world SKG datasets have proved the effectiveness of UniSKGRep in yielding general and substantial performance improvement compared with the strong baselines in various downstream tasks.

GraphCDA: a hybrid graph representation learning framework based on GCN and GAT for predicting disease-associated circRNAs.

CircularRNA (circRNA) is a class of noncoding RNA with high conservation and stability, which is considered as an important disease biomarker and drug target. Accumulating pieces of evidence have indicated that circRNA plays a crucial role in the pathogenesis and progression of many complex diseases. As the biological experiments are time-consuming and labor-intensive, developing an accurate computational prediction method has become indispensable to identify disease-related circRNAs. We presented a hybrid graph representation learning framework, named GraphCDA, for predicting the potential circRNA-disease associations. Firstly, the circRNA-circRNA similarity network and disease-disease similarity network were constructed to characterize the relationships of circRNAs and diseases, respectively. Secondly, a hybrid graph embedding model combining Graph Convolutional Networks and Graph Attention Networks was introduced to learn the feature representations of circRNAs and diseases simultaneously. Finally, the learned representations were concatenated and employed to build the prediction model for identifying the circRNA-disease associations. A series of experimental results demonstrated that GraphCDA outperformed other state-of-the-art methods on several public databases. Moreover, GraphCDA could achieve good performance when only using a small number of known circRNA-disease associations as the training set. Besides, case studies conducted on several human diseases further confirmed the prediction capability of GraphCDA for predicting potential disease-related circRNAs. In conclusion, extensive experimental results indicated that GraphCDA could serve as a reliable tool for exploring the regulatory role of circRNAs in complex diseases.