Link Prediction Task Research Articles

DFinder: A novel end-to-end graph embedding-based method to identify drug-food interactions.

Drug-food interactions (DFIs) occur when some constituents of food affect the bioaccessibility or efficacy of the drug by involving in drug pharmacodynamic (PD) and/or pharmacokinetic (PK) processes. Many computational methods have achieved remarkable results in link prediction tasks between biological entities which show the potential of computational methods in discovering novel DFIs. However, there are few computational approaches that pay attention to DFI identification. This is mainly due to the lack of drug-food interaction data. In addition, food is generally made up of a variety of chemical substances. The complexity of food makes it difficult to generate accurate feature representations for food. Therefore, it is urgent to develop effective computational approaches for learning the food feature representation and predicting drug-food interactions. In this paper, we first collect drug-food interaction data from DrugBank and PubMed respectively to construct two datasets, named DrugBank-DFI and PubMed-DFI. Based on these two datasets, two DFI networks are constructed. Then, we propose a novel end-to-end graph embedding-based method named DFinder to identify DFIs. DFinder combines node attribute features and topological structure features to learn the representations of drugs and food constituents. In topology space, we adopt a simplified graph convolution network-based method to learn the topological structure features. In feature space, we use a deep neural network to extract attribute features from the original node attributes. The evaluation results indicate that DFinder performs better than other baseline methods. The source code is available at https://github.com/23AIBox/23AIBox-DFinder. Supplementary data are available at Bioinformatics online.

Improving Graph Neural Network Models in Link Prediction Task via A Policy-Based Training Method

Graph neural network (GNN), as a widely used deep learning model in processing graph-structured data, has attracted numerous studies to apply it in the link prediction task. In these studies, observed edges in a network are utilized as positive samples, and unobserved edges are randomly sampled as negative ones. However, there are problems in randomly sampling unobserved edges as negative samples. First, some unobserved edges are missing edges that are existing edges in the network. Second, some unobserved edges can be easily distinguished from the observed edges, which cannot contribute sufficiently to the prediction task. Therefore, using the randomly sampled unobserved edges directly as negative samples is difficult to make GNN models achieve satisfactory prediction performance in the link prediction task. To address this issue, we propose a policy-based training method (PbTRM) to improve the quality of negative samples. In the proposed PbTRM, a negative sample selector generates the state vectors of the randomly sampled unobserved edges and determines whether to select them as negative samples. We perform experiments with three GNN models on two standard datasets. The results show that the proposed PbTRM can enhance the prediction performance of GNN models in the link prediction task.

Temporal knowledge graph embedding via sparse transfer matrix

Knowledge Graph Completion (KGC) is a fundamental problem for temporal knowledge graphs (TKGs), and TKGs embedding methods are one of the essential methods for KGC. However, existing TKG embedding methods encounter a scalability dilemma, i.e., the inconsistency in parameter scalability among different datasets, and the less use of global information, e.g., statistics and dependencies of facts. To mitigate these two issues, we propose a novel and effective TKG embedding method, named Temporal Knowledge Graph Embedding via Sparse Transfer Matrix (TASTER), which provides a framework to utilize both global and local information. Regarding a TKG as a static knowledge graph when ignoring the time dimension, TASTER first learns global embeddings based on this static knowledge graph to capture global information. To capture the local information in a specific timestamp, TASTER evolves local embeddings from global embeddings based on the corresponding subgraph. Besides, TASTER learns evolving entity embeddings through sparse transformation matrices, which could better adapt to TKGs with a varied number of subgraphs. We conduct experiments on three real-world datasets, and TASTER outperforms most existing models on the link prediction task of TKGs, which validates the its effectiveness.

Graph representation learning based on deep generative gaussian mixture models

Graph representation learning is an effective tool for facilitating graph analysis with machine learning methods. Most GNNs, including Graph Convolutional Networks (GCN), Graph Recurrent Neural Networks (GRNN), and Graph Auto-Encoders (GAE), employ vectors to represent nodes in a deterministic way without exploiting the uncertainty in hidden variables. Deep generative models are combined with GAE in the Variational Graph Auto-Encoder (VGAE) framework to address this issue. While traditional VGAE-based methods can capture hidden and hierarchical dependencies in latent spaces, they are limited by the data’s multimodality. Here, we propose the Gaussian Mixture Model (GMM) to model the prior distribution in VGAE. Furthermore, an adversarial regularization is incorporated into the proposed approach to ensure the fruitful impact of the latent representations on the results. We demonstrate the performance of the proposed method on clustering and link prediction tasks. Our experimental results on real datasets show remarkable performance compared to state-of-the-art methods.

Heterogeneous graph construction and HinSAGE learning from electronic medical records

Graph representation learning is a method for introducing how to effectively construct and learn patient embeddings using electronic medical records. Adapting the integration will support and advance the previous methods to predict the prognosis of patients in network models. This study aims to address the challenge of implementing a complex and highly heterogeneous dataset, including the following: (1) demonstrating how to build a multi-attributed and multi-relational graph model (2) and applying a downstream disease prediction task of a patient’s prognosis using the HinSAGE algorithm. We present a bipartite graph schema and a graph database construction in detail. The first constructed graph database illustrates a query of a predictive network that provides analytical insights using a graph representation of a patient’s journey. Moreover, we demonstrate an alternative bipartite model where we apply the model to the HinSAGE to perform the link prediction task for predicting the event occurrence. Consequently, the performance evaluation indicated that our heterogeneous graph model was successfully predicted as a baseline model. Overall, our graph database successfully demonstrated efficient real-time query performance and showed HinSAGE implementation to predict cardiovascular disease event outcomes on supervised link prediction learning.

Hypernetwork Representation Learning Based on Hyperedge Modeling

Most network representation learning approaches only consider the pairwise relationships between the nodes in ordinary networks but do not consider the tuple relationships, namely the hyperedges, among the nodes in the hypernetworks. Therefore, to solve the above issue, a hypernetwork representation learning approach based on hyperedge modeling, abbreviated as HRHM, is proposed, which fully considers the hyperedges to obtain ideal node representation vectors that are applied to downstream machine learning tasks such as node classification, link prediction, community detection, and so on. Experimental results on the hypernetwork datasets show that with regard to the node classification task, the mean node classification accuracy of HRHM approach goes beyond other best baseline approach by about 1% on the MovieLens and wordnet, and with regard to the link prediction task, except for HPHG approach, the mean AUC value of HRHM approach surpasses that of other baseline approaches by about 17%, 18%, and 6%, respectively, on the GPS, drug, and wordnet. The mean AUC value of HRHM approach is very close to that of other best baseline approach on the MovieLens.

A Two-Stage Deep-Learning Model for Link Prediction Based on Network Structure and Node Attributes

Link prediction, which is used to identify the potential relationship between nodes, is an important issue in network science. In existing studies, the traditional methods based on the structural similarity of nodes make it challenging to complete the task of link prediction in large-scale or sparse networks. Although emerging methods based on deep learning can solve this problem, most of the work mainly completes the link prediction through the similarity of the representation vector of network structure information. Many empirical studies show that link formation is affected by node attributes, and similarity is not the only criterion for the formation of links in reality. Accordingly, this paper proposed a two-stage deep-learning model for link prediction (i.e, TDLP), where the node representation vector of the network structure and attributes was obtained in the first stage, while link prediction was realized through supervised learning in the second stage. The empirical results on real networks showed that our model significantly outperforms the traditional methods (e.g., CN and RA), as well as newly proposed deep-learning methods (e.g., GCN and VGAE). This study not only proposed a deep-learning framework for link prediction from the perspective of structure and attribute fusion and link distribution capture, but also lays a methodological foundation for practical applications based on link prediction.

Learning attentive attribute-aware node embeddings in dynamic environments

Learning node embeddings is fundamental for numerous applications, such as link prediction and node classification. Node embeddings seek to learn a low-dimensional representation for each node in the graph. Many existing node representation learning methods for dynamic attributed graphs focus on preserving the temporal proximity of the nodes with relatively shallow models. However, real-life graphs are complex and usually exhibit evolutionary patterns of node attributes and graph structure. Therefore, the current state-of-the-art models fail to capture the information in the dynamic attributed graphs and settle for sub-optimal results. In this paper, we propose a novel model for embedding nodes in dynamic attributed graphs that captures the full extent of all relevant node information and the graph interrelations as well as graph evolutionary patterns. During model training, attribute-aware node embedding is learned using both graph and node properties in a dynamic context. Experiments demonstrate that our proposed method is superior to the state-of-the-art models in link prediction tasks. In addition, it introduces a novel way of learning richer representations by fully exploiting node attributes, graph structure, and evolutionary patterns in dynamic attributed graphs.

Attributed Graph Embedding Based on Attention with Cluster

Graph embedding is of great significance for the research and analysis of graphs. Graph embedding aims to map nodes in the network to low-dimensional vectors while preserving information in the original graph of nodes. In recent years, the appearance of graph neural networks has significantly improved the accuracy of graph embedding. However, the influence of clusters was not considered in existing graph neural network (GNN)-based methods, so this paper proposes a new method to incorporate the influence of clusters into the generation of graph embedding. We use the attention mechanism to pass the message of the cluster pooled result and integrate the whole process into the graph autoencoder as the third layer of the encoder. The experimental results show that our model has made great improvement over the baseline methods in the node clustering and link prediction tasks, demonstrating that the embeddings generated by our model have excellent expressiveness.

Integrating heterogeneous knowledge graphs into drug-drug interaction extraction from the literature.

Most of the conventional deep neural network-based methods for drug-drug interaction (DDI) extraction consider only context information around drug mentions in the text. However, human experts use heterogeneous background knowledge about drugs to comprehend pharmaceutical papers and extract relationships between drugs. Therefore, we propose a novel method that simultaneously considers various heterogeneous information for DDI extraction from the literature. We first construct drug representations by conducting the link prediction task on a heterogeneous pharmaceutical knowledge graph (KG) dataset. We then effectively combine the text information of input sentences in the corpus and the information on drugs in the heterogeneous KG (HKG) dataset. Finally, we evaluate our DDI extraction method on the DDIExtraction-2013 shared task dataset. In the experiment, integrating heterogeneous drug information significantly improves the DDI extraction performance, and we achieved an F-score of 85.40%, which results in state-of-the-art performance. We evaluated our method on the DrugProt dataset and improved the performance significantly, achieving an F-score of 77.9%. Further analysis showed that each type of node in the HKG contributes to the performance improvement of DDI extraction, indicating the importance of considering multiple pieces of information. Our code is available at https://github.com/tticoin/HKG-DDIE.git.

Relational graph convolutional networks: a closer look.

In this article, we describe a reproduction of the Relational Graph Convolutional Network (RGCN). Using our reproduction, we explain the intuition behind the model. Our reproduction results empirically validate the correctness of our implementations using benchmark Knowledge Graph datasets on node classification and link prediction tasks. Our explanation provides a friendly understanding of the different components of the RGCN for both users and researchers extending the RGCN approach. Furthermore, we introduce two new configurations of the RGCN that are more parameter efficient. The code and datasets are available at https://github.com/thiviyanT/torch-rgcn.

Predicting miRNA-lncRNA interactions on plant datasets based on bipartite network embedding method.

The research of miRNA-lncRNA interactions (MLIs) has received great attention recently due to their vital roles in microbiology and profound significance in diseases. Currently, many related studies mainly focus on animals and the link prediction problem on plants is rarely discussed comprehensively. Motivated by this, we achieve link prediction task based on the concept of bipartite graph and verify encouraging performance of our conclusions by conducting experiments on plant datasets. In this work, we firstly extract attribute information and structure information as base features and further process these information for network embedding. Intra-partition and inter-partition proximity modelling are conducted to construct the loss function, which facilitates the training of parameters. Finally, the superiority of our presented approach is shown by carrying out experiments on four plant datasets, which reflects the significance of this work to the research of microbiology and disease.

Masked prediction and interdependence network of the law using data from large-scale Japanese court judgments

Court judgments contain valuable information on how statutory laws and past court precedents are interpreted and how the interdependence structure among them evolves in the courtroom. Data-mining the evolving structure of such customs and norms that reflect myriad social values from a large-scale court judgment corpus is an essential task from both the academic and industrial perspectives. In this paper, using data from approximately 110,000 court judgments from Japan spanning the period 1998–2018 from the district to the supreme court level, we propose two tasks that grasp such a structure from court judgments and highlight the strengths and weaknesses of major machine learning models. One is a prediction task based on masked language modeling that connects textual information to legal codes and past court precedents. Another is a dynamic link prediction task where we predict the hidden interdependence structure in the law. We make quantitative and qualitative comparisons among major machine learning models to obtain insights for future developments.

Leveraging Representation Learning for the Construction and Application of a Knowledge Graph for Traditional Chinese Medicine: Framework Development Study.

BackgroundKnowledge discovery from treatment data records from Chinese physicians is a dramatic challenge in the application of artificial intelligence (AI) models to the research of traditional Chinese medicine (TCM).ObjectiveThis paper aims to construct a TCM knowledge graph (KG) from Chinese physicians and apply it to the decision-making related to diagnosis and treatment in TCM.MethodsA new framework leveraging a representation learning method for TCM KG construction and application was designed. A transformer-based Contextualized Knowledge Graph Embedding (CoKE) model was applied to KG representation learning and knowledge distillation. Automatic identification and expansion of multihop relations were integrated with the CoKE model as a pipeline. Based on the framework, a TCM KG containing 59,882 entities (eg, diseases, symptoms, examinations, drugs), 17 relations, and 604,700 triples was constructed. The framework was validated through a link predication task.ResultsExperiments showed that the framework outperforms a set of baseline models in the link prediction task using the standard metrics mean reciprocal rank (MRR) and Hits@N. The knowledge graph embedding (KGE) multitagged TCM discriminative diagnosis metrics also indicated the improvement of our framework compared with the baseline models.ConclusionsExperiments showed that the clinical KG representation learning and application framework is effective for knowledge discovery and decision-making assistance in diagnosis and treatment. Our framework shows superiority of application prospects in tasks such as KG-fused multimodal information diagnosis, KGE-based text classification, and knowledge inference–based medical question answering.

Item Relationship Graph Neural Networks for E-Commerce.

In a modern e-commerce recommender system, it is important to understand the relationships among products. Recognizing product relationships-such as complements or substitutes-accurately is an essential task for generating better recommendation results, as well as improving explainability in recommendation. Products and their associated relationships naturally form a product graph, yet existing efforts do not fully exploit the product graph's topological structure. They usually only consider the information from directly connected products. In fact, the connectivity of products a few hops away also contains rich semantics and could be utilized for improved relationship prediction. In this work, we formulate the problem as a multilabel link prediction task and propose a novel graph neural network-based framework, item relationship graph neural network (IRGNN), for discovering multiple complex relationships simultaneously. We incorporate multihop relationships of products by recursively updating node embeddings using the messages from their neighbors. An edge relational network is designed to effectively capture relational information between products. Extensive experiments are conducted on real-world product data, validating the effectiveness of IRGNN, especially on large and sparse product graphs.

A dynamic graph representation learning based on temporal graph transformer

The graph neural network has received significant attention in recent years because of its unique role in mining graph-structure data and its ubiquitous application in various fields, such as social networking and recommendation systems. Although most work focuses on learning low-dimensional node representation in static graphs, the dynamic nature of real-world networks makes temporal graphs more practical and significant. In this paper, we propose a dynamic graph representation learning method based on a temporal graph transformer (TGT), which can efficiently preserve high-order information and temporally evolve structural properties by incorporating an update module, an aggregation module, and a propagation module in a single model. The experimental results on three real-world networks demonstrate that the TGT outperforms state-of-the-art baselines for dynamic link prediction and edge classification tasks in terms of both accuracy and efficiency.

NEAWalk: Inferring missing social interactions via topological-temporal embeddings of social groups.

Real-world network data consisting of social interactions can be incomplete due to deliberately erased or unsuccessful data collection, which cause the misleading of social interaction analysis for many various time-aware applications. Naturally, the link prediction task has drawn much research interest to predict the missing edges in the incomplete social network. However, existing studies of link prediction cannot effectively capture the entangling topological and temporal dynamics already residing in the social network, thus cannot effectively reasoning the missing interactions in dynamic networks. In this paper, we propose the NEAWalk, a novel model to infer the missing social interaction based on topological-temporal features of patterns in the social group. NEAWalk samples the query-relevant walks containing both the historical and evolving information by focusing on the temporal constraint and designs a dual-view anonymization procedure for extracting both topological and temporal features from the collected walks to conduct the inference. Two-track experiments on several well-known network datasets demonstrate that the NEAWalk stably achieves superior performance against several state-of-the-art baseline methods.

Hypernetwork Representation Learning with Common Constraints of the Set and Translation

Different from conventional networks with only pairwise relationships among the nodes, there are also complex tuple relationships, namely the hyperedges among the nodes in the hypernetwork. However, most of the existing network representation learning methods cannot effectively capture the complex tuple relationships. Therefore, in order to resolve the above challenge, this paper proposes a hypernetwork representation learning method with common constraints of the set and translation, abbreviated as HRST, which incorporates both the hyperedge set associated with the nodes and the hyperedge regarded as the interaction relation among the nodes through the translation mechanism into the process of hypernetwork representation learning to obtain node representation vectors rich in the hypernetwork topology structure and hyperedge information. Experimental results on four hypernetwork datasets demonstrate that, for the node classification task, our method outperforms the other best baseline methods by about 1%. As for the link prediction task, our method is almost entirely superior to other baseline methods.

Collaborative representation learning for nodes and relations via heterogeneous graph neural network

Heterogeneous graphs, which consist of multiple types of nodes and edges, are highly suitable for characterizing real-world complex systems. In recent years, due to their strong capability of capturing rich semantics, heterogeneous graph neural networks (HGNNs) have proven to be a powerful technique for representation learning on heterogeneous graphs. However, most of the existing HGNNs only focus on learning node representations and ignore the learning of relation representations, which are complementary to node representations. To address this limitation, we propose a new HGNN model with Collaborative Representation Learning for Nodes and Relations (named CoNR) for link prediction task in this paper. Collaborative learning means that node representations and relation representations participate in and affect each other’s learning process. Specifically, node representations are obtained through a delicate two-step attention mechanism incorporating relation representations that can hierarchically aggregate information within one relation and across different relations. For relation representations, a relation encoder based on node information is designed to encode node representations into relation representations. Therefore, in this framework, node representations and relation representations are mutually updated in a layer-wise manner and work together to facilitate the downstream tasks better. Extensive experimental results on different datasets show the excellent performance of the proposed CoNR.

Time-Series Snapshot Network for Partner Recommendation: A Case Study on OSS

The last decade has witnessed the rapid growth of open-source software (OSS). Still, all contributors may find it difficult to assimilate into the OSS community even they are enthusiastic to make contributions. We thus suggest that partner recommendation across different roles may benefit both the users and developers, i.e., once we are able to make successful recommendation for those in need, it may dramatically contribute to the productivity of developers and the enthusiasm of users, thus further boosting OSS projects’ development. Motivated by this potential, we model the partner recommendation as link prediction task from email data via network embedding methods. In this article, we introduce time-series snapshot network (TSSN) that is a mixture network to model the interactions among users and developers. Based on the established TSSN, we perform temporal biased walk (TBW) to automatically capture both temporal and structural information of the email network, i.e., the behavioral similarity between individuals in the OSS email network. Experiments on ten Apache data sets demonstrate that the proposed TBW significantly outperforms a number of advanced random walk-based embedding methods, leading to the state-of-the-art recommendation performance.