Link Prediction Task Research Articles

Open Knowledge Graph Link Prediction with Semantic-Aware Embedding

Link prediction in open knowledge graphs (OpenKGs) is crucial for applications like question answering and recommendation systems. Existing OpenKG models leverage the semantic information of noun phrases (NPs) to enhance the performance in the link prediction task. However, these models only extract superficial semantic information from NPs, ignoring the fact that an NP possesses diverse semantics. Furthermore, these models have not fully exploited the semantic information of the relation phrases (RPs). To address these issues, we propose a model for link prediction called Open Knowledge Graph Link Prediction with Semantic-Aware Embedding (SeAE). First, we develop an adaptive disentanglement embedding (ADE) mechanism to learn the intrinsically abundant semantics of NPs. The ADE mechanism can adaptively calculate the embedding segmentation number according to the dataset and has an ingenious method for updating embeddings. Second, we integrate the attention mechanism into the GRU encoder to obtain the distribution of importance inside RP, facilitating a more comprehensive capture of the RP’s semantic information and enhancing the model’s interpretability. Finally, we design a relation gate, which extracts the RP semantic features of tail NP from the shared edge. This gate realizes the relation constraints on entities while enhancing the interaction between entities and relations. Extensive experiments on four benchmarks demonstrate that SeAE outperforms the state-of-the-art models, resulting in improvements of approximately 5.4% and 7.4% in MRR on ReVerb45K and ReVerb45KF datasets respectively.

A Multi-View Temporal Knowledge Graph Reasoning Framework with Interpretable Logic Rules and Feature Fusion

A temporal knowledge graph represents temporal information between entities in a multi-relational graph. Its reasoning aims to infer and predict potential links among entities. Predicting time-aware entities is a challenging task due to significant differences in entity appearances over time, such as different timestamps and frequencies. Current embedding-based similarity-matching methods have been introduced for predicting temporal facts. However, they lack deterministic logical explainability and cannot model the dynamic evolution of entities over time. To address these challenges, we propose a novel framework for temporal knowledge graph reasoning based on multi-view feature fusion (MVFF). First, MVFF extracts logical rules and uses the Gumbel-Softmax trick to sample high-quality rules. Second, it integrates logical rules, temporal quadruples, and factual triples to capture semantic features, temporal information, and structural information to solve link prediction tasks. Through experiments on four benchmark datasets, we show that MVFF outperforms state-of-the-art methods, providing not only better performance but also interpretable results.

GraphSAGE++: Weighted Multi-scale GNN for Graph Representation Learning

Graph neural networks (GNNs) have emerged as a powerful tool in graph representation learning. However, they are increasingly challenged by over-smoothing as network depth grows, compromising their ability to capture and represent complex graph structures. Additionally, some popular GNN variants only consider local neighbor information during node updating, ignoring the global structural information and leading to inadequate learning and differentiation of graph structures. To address these challenges, we introduce a novel graph neural network framework, GraphSAGE++. Our model extracts the representation of the target node at each layer and then concatenates all layer weighted representations to obtain the final result. In addition, the strategies combining double aggregations with weighted concatenation are proposed, which significantly enhance the model’s discernment and preservation of structural information. Empirical results on various datasets demonstrate that GraphSAGE++ excels in vertex classification, link prediction, and visualization tasks, surpassing existing methods in effectiveness.

A Radial Basis Function-Based Graph Attention Network With Squeeze Loss Optimization for Link Prediction

Graph attention networks is a popular method to deal with link prediction tasks, but the weight assigned to each sample is not focusing on the sample's own performance in training. Moreover, since the number of links is much larger than nodes in a graph, mapping functions are usually used to map the learned node features to link features, whose expression of node similarity determines the quality of link feature learning. To tackle the above issues, a new model graph attention networks based on Radial Basis Function (RBF) with squeeze loss is proposed, including two improvements. Firstly, RBF function with extended parameters is used to transform the node features output by attention layer into link features. The result of link feature embedding can be improved by shortening the distance of nodes with links and enlarging the distance of nodes without links in vector space. Secondly, squeeze loss is designed to adjust the loss according to the performance of samples in training and change the proportion of sample loss in the loss function to allocate training resources reasonably. The link prediction task performed on datasets shows that the performance of proposed method is better than baselines.

LUSTC: A Novel Approach for Predicting Link States on Dynamic Attributed Networks

Predicting link states on dynamic attributed networks is one of the most fundamental problems for network analysis. To accurately predict the formations and disappearances of links, we need to utilize three types of information, that is, structural information, node content information, and temporal information, simultaneously. To this end, we propose a novel approach called LUSTC for predicting links and unlinks with structural information, temporal information, and node content information on dynamic attributed networks. For each snapshot, LUSTC first randomly chooses some sets of active nodes whose degree changes between adjacent snapshots and collects higher-order structural information using a random walk method based on the active nodes. Then, LUSTC generates two global matrices containing the temporal structural information and temporal node content information, respectively, as well as a sequence of auxiliary matrices for reconstructing the structural information and node content information of snapshots. These generated matrices are optimized using a nonnegative matrix factorization (NMF) method based on the structural information and node content information of snapshots. Finally, LUSTC estimates the similarity matrix for future snapshots to predict the links that are more likely to be formed or broken. The experimental results on various dynamic attributed networks demonstrate the effectiveness of LUSTC on the link prediction and unlink prediction tasks.

Semantic-Enhanced Knowledge Graph Completion

Knowledge graphs (KGs) serve as structured representations of knowledge, comprising entities and relations. KGs are inherently incomplete, sparse, and have a strong need for completion. Although many knowledge graph embedding models have been designed for knowledge graph completion, they predominantly focus on capturing observable correlations between entities. Due to the sparsity of KGs, potential semantic correlations are challenging to capture. To tackle this problem, we propose a model entitled semantic-enhanced knowledge graph completion (SE-KGC). SE-KGC effectively addresses the issue by incorporating predefined semantic patterns, enabling the capture of semantic correlations between entities and enhancing features for representation learning. To implement this approach, we employ a multi-relational graph convolution network encoder, which effectively encodes the KG. Subsequently, we utilize a scoring decoder to evaluate triplets. Experimental results demonstrate that our SE-KGC model outperforms other state-of-the-art methods in link-prediction tasks across three datasets. Specifically, compared to the baselines, SE-KGC achieved improvements of 11.7%, 1.05%, and 2.30% in terms of MRR on these three datasets. Furthermore, we present a comprehensive analysis of the contributions of different semantic patterns, and find that entities with higher connectivity play a pivotal role in effectively capturing and characterizing semantic information.

Hypergraph modeling and hypergraph multi-view attention neural network for link prediction

Hypergraph neural networks are widely used in link prediction because of their ability to learn the high-order structure relationship. However, most existing hypergraph modeling relies on the attribute information of nodes. And as for the link prediction, missing links are not utilized when training link predictors, so conventional transductive hypergraph learning are generally not consistent with link prediction tasks. To address these limitations, we propose the Network Structure Linear Representation (NSLR) method to model hypergraph for general networks without node attribute information and the inductive hypergraph learning method Hypergraph Multi-view Attention Neural Network (HMANN) that learns the rich high-order structure information from node-level and hyperedge-level. Also, this paper put forwards a novel NSLR-HMANN link prediction algorithm based on NSLR and HMANN methods. Extensive comparison and ablation experiments show that the NSLR-HMANN link prediction algorithm achieves state-of-the-art performance on link prediction and has better performance on robustness.

Community-Aware Evolution Similarity for Link Prediction in Dynamic Social Networks

The link prediction problem is a time-evolving model in network science that has simultaneously abetted myriad applications and experienced extensive methodological improvement. Inferring the possibility of emerging links in dynamic social networks, also known as the dynamic link prediction task, is complex and challenging. In contrast to the link prediction in cross-sectional networks, dynamic link prediction methods need to cater to the actor-level temporal changes and associated evolutionary information regarding their micro- (i.e., link formation/deletion) and mesoscale (i.e., community formation) network structure. With the advent of abundant community detection algorithms, the research community has examined community-aware link prediction strategies in static networks. However, the same task in dynamic networks where, apart from the actors and links among them, their community pattern is also dynamic, is yet to be explored. Evolutionary community-aware information, including the associated link structure and temporal neighborhood changes, can effectively be mined to build dynamic similarity metrics for dynamic link prediction. This study aims to develop and integrate such dynamic features with machine learning algorithms for link prediction tasks in dynamic social networks. It also compares the performances of these features against well-known similarity metrics (i.e., ResourceAllocation) for static networks and a time series-based link prediction strategy in dynamic networks. These proposed features achieved high-performance scores, representing them as prospective candidates for both dynamic link prediction tasks and modeling the network growth.

Relation-oriented few-shot knowledge graph prototype networks

Knowledge graph (KG) embedding has been widely researched, but it suffers from some problems in the few-shot scenarios. The topological and semantic connection among entities cannot satisfy the assumption of samples’ independent identically distribution. Additionally, these relations between entity pairs are various and complex. But most of the previous methods are node-oriented, which is weak in modeling complex relations. In order to solve the above problems, we propose a few-shot relation prototype network (FRPN). In our method, each relation is regarded as a learning object, instead of just entity pairs’ concatenation. A relation-oriented two-channel embedding mechanism is designed to achieve multi-scale information aggregation at the entity level and the relation level respectively. For the entity level, it aggregates information at different scales according to the relation type and the neighbors under each relation. For the relation level, our model obtains each kind of relation’s prototype from the ontology and the entity layer. The multi-scale aggregation contributes to KG embedding in the few-shot scenario. Compared with existing models, our model has significantly improved the performance on both the link prediction and triple classification tasks in two few-shot datasets.

Knowledge graph embedding based on dynamic adaptive atrous convolution and attention mechanism for link prediction

Knowledge graph embedding (KGE) is essential for various applications, particularly in link prediction and other downstream tasks. While existing convolutional neural network (CNN)-based methods have been effective, they face challenges in comprehensively capturing local and global contextual information from triplets. To address this challenge, we propose a KGE method based on a dynamic adaptive atrous convolution and attention mechanism (DTAE). In this study, we innovatively construct adaptive convolutional kernels derived from relation representations. Employing a parallel strategy, we apply a multidimensional attention mechanism across kernel space. This configuration allows our kernels to dynamically adapt and learn from different entity representations, ensuring more nuanced feature extraction. Moreover, a dynamic adaptive atrous convolutional network has been developed. This network leverages atrous convolution, efficiently broadening the receptive field to encompass global information while preserving essential details during the aggregation process. In addition, we enhance feature information by integrating a channel attention mechanism, further refining our model’s ability to discern and process relevant data. Via comprehensive experiments on five standard link prediction benchmark datasets, DTAE demonstrates superior performance over existing models. Compared to the best recent state-of-the-art models over the last three years, we achieve improvements across various metrics ranging from 0.4% to 14.4%.

BehaviorNet: A Fine-grained Behavior-aware Network for Dynamic Link Prediction

Dynamic link prediction has become a trending research subject because of its wide applications in the web, sociology, transportation, and bioinformatics. Currently, the prevailing approach for dynamic link prediction is based on graph neural networks, in which graph representation learning is the key to perform dynamic link prediction tasks. However, there are still great challenges because the structure of graphs evolves over time. A common approach is to represent a dynamic graph as a collection of discrete snapshots, in which information over a period is aggregated through summation or averaging. This way results in some fine-grained time-related information loss, which further leads to a certain degree of performance degradation. We conjecture that such fine-grained information is vital because it implies specific behavior patterns of nodes and edges in a snapshot. To verify this conjecture, we propose a novel fine-grained behavior-aware network (BehaviorNet) for dynamic network link prediction. Specifically, BehaviorNet adapts a transformer-based graph convolution network to capture the latent structural representations of nodes by adding edge behaviors as an additional attribute of edges. GRU is applied to learn the temporal features of given snapshots of a dynamic network by utilizing node behaviors as auxiliary information. Extensive experiments are conducted on several real-world dynamic graph datasets, and the results show significant performance gains for BehaviorNet over several state-of-the-art (SOTA) discrete dynamic link prediction baselines. Ablation study validates the effectiveness of modeling fine-grained edge and node behaviors.

A few-shot link prediction framework to drug repurposing using multi-level attention network

Drug repurposing is a strategy aiming at uncovering novel medical indications of approved drugs. This process of discovery can be effectively represented as a link prediction task within a medical knowledge graph by predicting the missing relation between the disease entity and the drug entity. Typically, the links to be predicted pertain to rare types, thereby necessitating the task of few-shot link prediction. However, the sparsity of neighborhood information and weak triplet interactions result in less effective representations, which brings great challenges to the few-shot link prediction. Therefore, in this paper, we proposed a meta-learning framework based on a multi-level attention network (MLAN) to capture valuable information in the few-shot scenario for drug repurposing. First, the proposed method utilized a gating mechanism and a graph attention network to effectively filter noise information and highlight the valuable neighborhood information, respectively. Second, the proposed commonality relation learner, employing a set transformer, effectively captured triplet-level interactions while remaining insensitive to the size of the support set. Finally, a model-agnostic meta-learning training strategy was employed to optimize the model quickly on each meta task. We conducted validation of the proposed method on two datasets specifically designed for few-shot link prediction in medical field: COVID19-One and BIOKG-One. Experimental results showed that the proposed model had significant advantages over state-of-the-art few-shot link prediction methods. Results also highlighted the valuable insights of the proposed method, which successfully integrated the components within a unified meta-learning framework for drug repurposing.

Knowledge graph embeddings in the biomedical domain: are they useful? A look at link prediction, rule learning, and downstream polypharmacy tasks.

Knowledge graphs (KGs) are powerful tools for representing and organizing complex biomedical data. They empower researchers, physicians, and scientists by facilitating rapid access to biomedical information, enabling the discernment of patterns or insights, and fostering the formulation of decisions and the generation of novel knowledge. To automate these activities, several KG embedding algorithms have been proposed to learn from and complete KGs. However, the efficacy of these embedding algorithms appears limited when applied to biomedical KGs, prompting questions about whether they can be useful in this field. To that end, we explore several widely used KG embedding models and evaluate their performance and applications using a recent biomedical KG, BioKG. We also demonstrate that by using recent best practices for training KG embeddings, it is possible to improve performance over BioKG. Additionally, we address interpretability concerns that naturally arise with such machine learning methods. In particular, we examine rule-based methods that aim to address these concerns by making interpretable predictions using learned rules, achieving comparable performance. Finally, we discuss a realistic use case where a pretrained BioKG embedding is further trained for a specific task, in this case, four polypharmacy scenarios where the goal is to predict missing links or entities in another downstream KGs in four polypharmacy scenarios. We conclude that in the right scenarios, biomedical KG embeddings can be effective and useful. Our code and data is available at https://github.com/aryopg/biokge.

Dual-Safety Knowledge Graph Completion for Process Industry

With the rise of Industry 4.0, control systems have taken on increasing importance in industrial processes, and ensuring their security has become a pressing issue. While recent research has focused on cybersecurity threats, the security risks inherent to industrial processes themselves have been overlooked. Additionally, existing tools cannot simultaneously analyze both cyber vulnerabilities and processes anomaly in industrial settings. This paper aims to address these issues through two main contributions. First, we develop a knowledge graph to integrate information on security risks across cybersecurity and industrial processes, providing a foundation for comprehensively assessing threats. Second, we apply the link prediction task to the knowledge graph, introducing an embedding-based approach to unveil previously undiscovered knowledge. Our experiments demonstrate that the proposed method exhibits comparable performance on link prediction and is capable of mining valuable and diverse potential risks in industrial processes.

Improving Landslide Prediction: Innovative Modeling and Evaluation of Landslide Scenario with Knowledge Graph Embedding

The increasing frequency and magnitude of landslides underscore the growing importance of landslide prediction in light of factors like climate change. Traditional methods, including physics-based methods and empirical methods, are beset by high costs and a reliance on expert knowledge. With the advancement of remote sensing and machine learning, data-driven methods have emerged as the mainstream in landslide prediction. Despite their strong generalization capabilities and efficiency, data-driven methods suffer from the loss of semantic information during training due to their reliance on a ‘sequence’ modeling method for landslide scenarios, which impacts their predictive accuracy. An innovative method for landslide prediction is proposed in this paper. In this paper, we propose an innovative landslide prediction method. This method designs the NADE ontology as the schema layer and constructs the data layer of the knowledge graph, utilizing tile lists, landslide inventory, and environmental data to enhance the representation of complex landslide scenarios. Furthermore, the transformation of the landslide prediction task into a link prediction task is carried out, and a knowledge graph embedding model is trained to achieve landslide predictions. Experimental results demonstrate that the method improves the F1 score by 5% in scenarios with complete datasets and 17% in scenarios with sparse datasets compared to data-driven methods. Additionally, the application of the knowledge graph embedding model is utilized to generate susceptibility maps, and an analysis of the effectiveness of entity embeddings is conducted, highlighting the potential of knowledge graph embeddings in disaster management.

Local structure-aware graph contrastive representation learning

Traditional Graph Neural Network (GNN), as a graph representation learning method, is constrained by label information. However, Graph Contrastive Learning (GCL) methods, which tackles the label problem effectively, mainly focus on the feature information of the global graph or small subgraph structure (e.g., the first-order neighborhood). In this paper, we propose a Local Structure-aware Graph Contrastive representation Learning method (LS-GCL) to model the structural information of nodes from multiple views. Specifically, we construct the semantic subgraphs that are not limited to the first-order neighbors. For the local view, the semantic subgraph of each target node is input into a shared GNN encoder to obtain the target node embeddings at the subgraph-level. Then, we use a pooling function to generate the subgraph-level graph embeddings. For the global view, considering the original graph preserves indispensable semantic information of nodes, we leverage the shared GNN encoder to learn the target node embeddings at the global graph-level. The proposed LS-GCL model is optimized to maximize the common information among similar instances at three various perspectives through a multi-level contrastive loss function. Experimental results on six datasets illustrate that our method outperforms state-of-the-art graph representation learning approaches for both node classification and link prediction tasks.

SDFormer: A shallow-to-deep feature interaction for knowledge graph embedding

Inferring missing information from current facts in a knowledge graph (KG) is the target of the link prediction task. Currently, existing methods embed the entities and relations of KG as a whole into a low-dimensional vector space. Nonetheless, they ignore the multi-level interactions (shallow interactions, deep interactions) among the finer-grained sub-features of entities and relations. To overcome these limitations, we present a shallow-to-deep feature interaction for knowledge graph embedding (SDFormer). It takes into account the interpretability of sub-feature tokens of entities and relations and learns shallow-to-deep interaction information between entities and relations at a more fine-grained level. Specifically, entity and relation vectors are decomposed into sub-features to represent multi-dimensional information. Then, a shallow-to-deep feature interaction method is designed to capture multi-level interactions between entities and relations. This process enriches the feature representation by modeling the interaction between sub-features. Finally, a 1-X scoring function is utilized to calculate the score of each knowledge triplet. The experimental results on several benchmark datasets show that SDFormer obtains competitive performance results and more efficient training efficiency on other comparative models and because of the shallow-to-deep feature interaction between entities and relations.

DeepWalk with Reinforcement Learning (DWRL) for node embedding

DeepWalk is used to convert nodes in an original graph into equivalent vectors in a latent space for performing various predictive tasks. To ensure second-order structural similarity between nodes in the original graph and their vectors in the latent space, dot products are applied to each pair of nodes explored on the random walk (RW) in the latent space. However. dot products for graphs with millions of nodes and billions of edges are computationally expensive. To minimize the computation time required for calculating the second-order structural similarity, DeepWalk with reinforcement learning (DWRL) is proposed herein. In DWRL, a level pointer for each node in the original graph is prepared. By identifying common nodes between each pair of nodes in the original graph, the number of computations in the dot product in the latent space is reduced, thereby ensuring second-order structural similarity. Additionally, repeated selection of the same node during RWs produces redundant samples for training. Therefore, the subsampling technique is used to choose the next node based on its degree, which improves the generalization of node representations in the latent space and increases accuracy. The proposed techniques are applied to popular datasets to perform multilabel classification and link prediction tasks, and their efficiency in reducing the computation time is verified. The proposed DWRL minimizes the computation time 47% for large graphs to build latent vectors and improves the average micro and macro F1 scores up to 12%. The link prediction performance also increases up to 20%.

TDN: Triplet Distributor Network for Knowledge Graph Completion

Conventional Knowledge Graph Completion (KGC) methods typically map entities and relations to a unified space through the shared mapping matrix, and then interact with entities and relations to infer the missing items in the knowledge graph. Although this shared mapping matrix considers the suitability of all triplets, it neglects the specificity of each triplet. To solve this problem, we dynamically learn one information distributor for each triplet to exchange its specific information. In this paper, we propose a novel Triplet Distributor Network (TDN) for the knowledge graph completion task. Specifically, we adaptively learn one Triplet Distributor (TD) for each triplet to assist the interaction between the entity and relation. Furthermore, on the basis of TD, we creatively design the information exchange layer to dynamically propagate the information of the entity and relation, thus mutually enhancing entity and relation representations. Except for several commonly-used knowledge graph datasets, we still implement the link prediction task on the social-relational and medical datasets to test the proposed method. Experimental results demonstrate that the proposed method performs better than existing state-of-the-art KGC methods. The source codes of this paper are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/TDN for Knowledge Graph Completion.git</uri> .

Entity and Entity Type Composition Representation Learning for Knowledge Graph Completion

This paper proposes a simple knowledge graph embedding (KGE) framework that considers the entity type information without additional resources. The KGE is used to obtain vector representations of entities and relations by learning structured information in triples. The obtained vectors are used to predict the missing links in a knowledge graph (KG). Although many KGs contain entity type information, most of the existing methods ignored the potential of the entity type information for the link prediction task. The proposed framework, which is called entity and entity type composition representation learning (EETCRL), obtains vector representations of both entities and entity types, which are combined and used for link prediction. Experimental results on three datasets show that the EETCRL outperforms the baseline methods in most cases. Furthermore, the results obtained from tests with different model sizes show that the proposed framework can achieve high performance even with a small model size. This paper also discusses the effect of considering information about entity types on the link prediction task by analyzing the experimental results.