Knowledge Graph Completion Task Research Articles

Disentangled Relational Graph Neural Network with Contrastive Learning for knowledge graph completion

Learning disentangled entity representations has garnered significant attention in the field of knowledge graph completion (KGC). However, the existing methods inherently overlook the indicative role of relations and the correlation between latent factors and relations, leading to suboptimal entity representations for KGC tasks. In the current study, we introduce the Disentangled Relational Graph Neural Network with Contrastive Learning (DRGCL) method, designed to acquire disentangled entity representations guided by relations. In particular, we first devise the factor-aware relational message aggregation approach to learn entity representations under each semantic subspace and obtain latent factor representations by attention mechanisms. Subsequently, we propose a discrimination objective for factor-subspace pairs using a contrastive learning approach, which compels the factor representations to distinctly capture the information associated with different latent factors and promote the consistency between factor representations and semantic subspaces. Through disentanglement, our model can generate relation-aware scores tailored to the provided scenario. Extensive experiments have been conducted on three benchmark datasets and the results demonstrate the superiority of our method compared with strong baseline models.

Neuro-Symbolic Integration for Reasoning and Learning on Knowledge Graphs

The goal of this thesis is to address knowledge graph completion tasks using neuro-symbolic methods. Neuro-symbolic methods allow the joint utilization of symbolic information defined as meta-rules in ontologies and knowledge graph embedding methods that represent entities and relations of the graph in a low-dimensional vector space. This approach has the potential to improve the resolution of knowledge graph completion tasks in terms of reliability, interpretability, data-efficiency and robustness.

Mixed Geometry Message and Trainable Convolutional Attention Network for Knowledge Graph Completion

Knowledge graph completion (KGC) aims to study the embedding representation to solve the incompleteness of knowledge graphs (KGs). Recently, graph convolutional networks (GCNs) and graph attention networks (GATs) have been widely used in KGC tasks by capturing neighbor information of entities. However, Both GCNs and GATs based KGC models have their limitations, and the best method is to analyze the neighbors of each entity (pre-validating), while this process is prohibitively expensive. Furthermore, the representation quality of the embeddings can affect the aggregation of neighbor information (message passing). To address the above limitations, we propose a novel knowledge graph completion model with mixed geometry message and trainable convolutional attention network named MGTCA. Concretely, the mixed geometry message function generates rich neighbor message by integrating spatially information in the hyperbolic space, hypersphere space and Euclidean space jointly. To complete the autonomous switching of graph neural networks (GNNs) and eliminate the necessity of pre-validating the local structure of KGs, a trainable convolutional attention network is proposed by comprising three types of GNNs in one trainable formulation. Furthermore, a mixed geometry scoring function is proposed, which calculates scores of triples by novel prediction function and similarity function based on different geometric spaces. Extensive experiments on three standard datasets confirm the effectiveness of our innovations, and the performance of MGTCA is significantly improved compared to the state-of-the-art approaches.

Progressive Distillation Based on Masked Generation Feature Method for Knowledge Graph Completion

In recent years, knowledge graph completion (KGC) models based on pre-trained language model (PLM) have shown promising results. However, the large number of parameters and high computational cost of PLM models pose challenges for their application in downstream tasks. This paper proposes a progressive distillation method based on masked generation features for KGC task, aiming to significantly reduce the complexity of pre-trained models. Specifically, we perform pre-distillation on PLM to obtain high-quality teacher models, and compress the PLM network to obtain multi-grade student models. However, traditional feature distillation suffers from the limitation of having a single representation of information in teacher models. To solve this problem, we propose masked generation of teacher-student features, which contain richer representation information. Furthermore, there is a significant gap in representation ability between teacher and student. Therefore, we design a progressive distillation method to distill student models at each grade level, enabling efficient knowledge transfer from teachers to students. The experimental results demonstrate that the model in the pre-distillation stage surpasses the existing state-of-the-art methods. Furthermore, in the progressive distillation stage, the model significantly reduces the model parameters while maintaining a certain level of performance. Specifically, the model parameters of the lower-grade student model are reduced by 56.7\% compared to the baseline.

Learnable convolutional attention network for knowledge graph completion

Recently, graph convolutional networks (GCNs) and graph attention networks (GATs) have been used extensively in knowledge graph completion (KGC), which aims to solve the incompleteness of knowledge graphs (KGs). However, both GCNs and GATs have limitations in the KGC task, and the best method is analyzing the neighbors of each entity (pre-validating), while this process is prohibitively expensive. Furthermore, relations in KGs have specific semantics and should be considered when aggregating neighbor information (message passing). To address the above limitations, we propose a learnable convolutional attention network for knowledge graph completion named LCA-KGC. LCA-KGC introduces a knowledge graph convolutional attention network using a convolution operation before the attention mechanism to ensure structural information acquisition and avoid redundant information stacking. Furthermore, to complete the autonomous switching of GNNs types and eliminate the necessity of pre-validating the local structure of KGs, LCA-KGC designs a learnable knowledge graph convolutional attention network by comprising three types of GNNs in one learnable formulation. Moreover, a learnable message function is proposed to emphasize relational semantics when aggregating neighbor information. Extensive experiments on standard KG datasets validate the effectiveness of the proposed innovations, and LCA-KGC achieves state-of-the-art (SOTA) performance compared to existing approaches (e.g., compared to SOTA approaches, LCA-KGC improves MRR from 0.360 to 0.372 on FB15k-237 dataset, and Hits@3 from 0.561 to 0.581 on YAGO3-10 dataset).

TDN: Triplet Distributor Network for Knowledge Graph Completion

Conventional Knowledge Graph Completion (KGC) methods typically map entities and relations to a unified space through the shared mapping matrix, and then interact with entities and relations to infer the missing items in the knowledge graph. Although this shared mapping matrix considers the suitability of all triplets, it neglects the specificity of each triplet. To solve this problem, we dynamically learn one information distributor for each triplet to exchange its specific information. In this paper, we propose a novel Triplet Distributor Network (TDN) for the knowledge graph completion task. Specifically, we adaptively learn one Triplet Distributor (TD) for each triplet to assist the interaction between the entity and relation. Furthermore, on the basis of TD, we creatively design the information exchange layer to dynamically propagate the information of the entity and relation, thus mutually enhancing entity and relation representations. Except for several commonly-used knowledge graph datasets, we still implement the link prediction task on the social-relational and medical datasets to test the proposed method. Experimental results demonstrate that the proposed method performs better than existing state-of-the-art KGC methods. The source codes of this paper are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/TDN for Knowledge Graph Completion.git</uri> .

MEGA: Meta-Graph Augmented Pre-Training Model for Knowledge Graph Completion

Nowadays, a large number of Knowledge Graph Completion (KGC) methods have been proposed by using embedding based manners, to overcome the incompleteness problem faced with knowledge graph (KG). One important recent innovation in Natural Language Processing (NLP) domain is the employ of deep neural models that make the most of pre-training, culminating in BERT, the most popular example of this line of approaches today. Recently, a series of new KGC methods introducing a pre-trained language model, such as KG-BERT, have been developed and released compelling performance. However, previous pre-training based KGC methods usually train the model by using simple training task and only utilize one-hop relational signals in KG, which leads that they cannot model high-order semantic contexts and multi-hop complex relatedness. To overcome this problem, this paper presents a novel pre-training framework for KGC task, which especially consists of both one-hop relation level task (low-order) and multi-hop meta-graph level task (high-order). Hence, the proposed method can capture not only the elaborate sub-graph structure but also the subtle semantic information on the given KG. The empirical results show the efficiency of the proposed method on the widely-used real-world datasets.

Simple Knowledge Graph Completion Model Based on Differential Negative Sampling and Prompt Learning

Knowledge graphs (KGs) serve as a crucial resource for numerous artificial intelligence tasks, significantly contributing to the advancement of the AI field. However, the incompleteness of existing KGs hinders their effectiveness in practical applications. Consequently, researchers have proposed the task of KG completion. Currently, embedding-based techniques dominate the field as they leverage the structural information within KGs to infer and complete missing parts. Nonetheless, these methods exhibit limitations. They are limited by the quality and quantity of structural information and are unable to handle the missing entities in the original KG. To overcome these challenges, researchers have attempted to integrate pretrained language models and textual data to perform KG completion. This approach utilizes the definition statements and description text of entities within KGs. The goal is to compensate for the latent connections that are difficult for traditional methods to obtain. However, text-based methods still lag behind embedding-based models in terms of performance. Our analysis reveals that the critical issue lies in the selection process of negative samples. In order to enhance the performance of the text-based methods, various types of negative sampling methods are employed in this study. We introduced prompt learning to fill the gap between the pre-training language model and the knowledge graph completion task, and to improve the model reasoning level. Simultaneously, a ranking strategy based on KG structural information is proposed to utilize KG structured data to assist reasoning. The experiment results demonstrate that our model exhibits strong competitiveness and outstanding inference speed. By fully exploiting the internal structural information of KGs and external relevant descriptive text resources, we successfully elevate the performance levels of KG completion tasks across various metrics.

KE-X: Towards subgraph explanations of knowledge graph embedding based on knowledge information gain

Over the past years, knowledge graph embedding approaches have proven effective for knowledge graph completion tasks. However, most existing models are either built on a certain embedding space or black-box neural networks, making it hard to access explanations for prediction results and resulting in limited explainability. In this paper, we propose to leverage information entropy to quantify the importance of explanation candidates, then build a framework KE-X, for explaining results from knowledge graph embedding approaches by generating explainable subgraphs. Specifically, by performing a modified message passing mechanism on a partially masked knowledge subgraph and maximizing knowledge information gain, KE-X can extract the most valuable subgraph explanation for a link prediction query. To evaluate KE-X, we conduct experiments on three real-world knowledge graphs with two representative KGE models, TransE and DistMult. Both quantitative and case study results show that our framework can extract high-quality explanations.

EARR: Using rules to enhance the embedding of knowledge graph

Knowledge graphs have been receiving increasing attention from researchers. However, most of these graphs are incomplete, leading to the rise of knowledge graph completion as a prominent task. The goal of knowledge graph completion is to find missing relations in a knowledge graph. Knowledge graph embedding represents the entities and relations in a low-dimensional embedding space, simplifying operations and allowing for integration with knowledge graph completion tasks. Several popular embedding models, such as TransE, TransH, TransR, TuckER, RotatE, and others have achieved impressive results on knowledge graph completion tasks. However, most of these methods do not incorporate background knowledge that could enhance the quality of knowledge embedding. Logic rules are adaptable and scalable, which can enrich background knowledge, and separating the attributes of entities can improve the relevance of relations and facilitate the accuracy of logic rule extraction. Thus, we propose a novel method, named Entity–Attribute–Relation–Rule (EARR), which separates attributes from entities and uses logic rules to extend the dataset, improving the accuracy of knowledge graph completion tasks. We define a total of six rules in this paper, including Rule 1–3, Rule 5, and Rule 6 for entities, and Rule 4 for entities and attributes. We evaluate our method based on the task of link prediction through two kinds of experiments. In the basic experiment, we compare our method with three benchmark models, namely, TransE, TransH, and TransR. In the experiment with different size datasets, FB24K and CoDEx, we evaluate our method on different size datasets with different models, including TransE, TuckER, and RotatE. The experimental results indicate that EARR can improve the quality of knowledge graph embedding.

Zero-Shot and Few-Shot Learning With Knowledge Graphs: A Comprehensive Survey

Machine learning (ML), especially deep neural networks, has achieved great success, but many of them often rely on a number of labeled samples for supervision. As sufficient labeled training data are not always ready due to, e.g., continuously emerging prediction targets and costly sample annotation in real-world applications, ML with sample shortage is now being widely investigated. Among all these studies, many prefer to utilize auxiliary information including those in the form of knowledge graph (KG) to reduce the reliance on labeled samples. In this survey, we have comprehensively reviewed over <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{90}$</tex-math> </inline-formula> articles about KG-aware research for two major sample shortage settings—zero-shot learning (ZSL) where some classes to be predicted have no labeled samples and few-shot learning (FSL) where some classes to be predicted have only a small number of labeled samples that are available. We first introduce KGs used in ZSL and FSL as well as their construction methods and then systematically categorize and summarize KG-aware ZSL and FSL methods, dividing them into different paradigms, such as the mapping-based, the data augmentation, the propagation-based, and the optimization-based. We next present different applications, including not only KG augmented prediction tasks such as image classification, question answering, text classification, and knowledge extraction but also KG completion tasks and some typical evaluation resources for each task. We eventually discuss some challenges and open problems from different perspectives.

ResE: A Fast and Efficient Neural Network-Based Method for Link Prediction

In this study, we present a novel embedding model, named ResE, for predicting links in knowledge graphs. ResE employs depth-separable convolution and residual blocks, integrated with channel attention mechanisms. ResE surpasses previously published models, including the closely related TransE model, by achieving the satisfactory mean rank (MR) and the excellent Hits@10 scores on both WN18RR and FB15K-237 benchmarks. ResE is a promising model for knowledge graph completion tasks, with potential for further investigation and extension to new applications such as user-oriented relationship modeling. Although comparatively shallow compared to computer vision convolutional architectures, future work may explore deeper convolutional models. ResE exhibits remarkable performance and outperforms existing approaches, thus setting a new benchmark for knowledge graph completion. The outcomes of our study illustrate the effectiveness of incorporating depth-separable convolution and residual blocks, accompanied by channel attention mechanisms, in modeling knowledge graphs. These findings highlight ResE’s potential to push the boundaries of cutting-edge in this domain.

Multi-Concept Representation Learning for Knowledge Graph Completion

Knowledge Graph Completion (KGC) aims at inferring missing entities or relations by embedding them in a low-dimensional space. However, most existing KGC methods generally fail to handle the complex concepts hidden in triplets, so the learned embeddings of entities or relations may deviate from the true situation. In this article, we propose a novel M ulti- c oncept R epresentation L earning (McRL) method for the KGC task, which mainly consists of a multi-concept representation module, a deep residual attention module, and an interaction embedding module. Specifically, instead of the single-feature representation, the multi-concept representation module projects each entity or relation to multiple vectors to capture the complex conceptual information hidden in them. The deep residual attention module simultaneously explores the inter- and intra-connection between entities and relations to enhance the entity and relation embeddings corresponding to the current contextual situation. Moreover, the interaction embedding module further weakens the noise and ambiguity to obtain the optimal and robust embeddings. We conduct the link prediction experiment to evaluate the proposed method on several standard datasets, and experimental results show that the proposed method outperforms existing state-of-the-art KGC methods.

Multilingual Knowledge Graph Completion Based on Structure Features of the Dual-Branch

With the development of information fusion, knowledge graph completion tasks have received a lot of attention.​ some studies investigate the broader underlying problems of linguistics, while embedding learning has a narrow focus.​This poses significant challenges due to the heterogeneity of coarse-graining patterns. Then, to settle the whole matter,​ a framework for completion is designed, named Triple Encoder-Scoring Module (TEsm). The model employs an alternating two-branch structure that fuses local features into the interaction pattern of the triplet itself by perfectly combining distance and structure models. Moreover, it is mapped to a uniform shared space. Upon completion, an ensemble inference method is proposed to query multiple predictions from different graphs using a weight classifier. Experiments show that the experimental dataset used for the completion task is DBpedia, which contains five different linguistic subsets..​Our extensive experimental results demonstrate that TEsm can efficiently and smoothly solve the optimal completion task, validating the performance of the proposed model.​

Temporal-structural importance weighted graph convolutional network for temporal knowledge graph completion

Knowledge graph completion (KGC) tasks are aimed to reason out missing facts in a knowledge graph. However, knowledge often evolves over time, and static knowledge graph completion methods have difficulty in identifying its changes. Scholars have focus on temporal knowledge graph completion (TKGC). Most existing TKGC methods incorporate temporal information into triples and convert them into KGC tasks, ignoring the impact of temporal information on quaternions. Furthermore, existing embedding learning methods based on message-passing network aggregate features passed by neighbors with the same attention, ignoring the complex structure information that each node has different importance in passing the message. Therefore, to capture the impact of temporal information on quaternions and structural information on nodes, we proposed a TKGC method based on temporal attention learning (TAL-TKGC), which includes a temporal attention module and an importance-weighted GCN. The temporal attention module was designed to capture the deep connection between timestamps entities and relations at the semantic levels. The importance-weighted GCN considers the structural importance and attention of temporal information to entities for weighted aggregation. Finally, we conducted experiments on two public datasets, and the results proved the performance of our method. We also performed the speedup experiments in a distributed environment, and the proposed model has an excellent scalability on multiple GPUs.

FLONE: fully Lorentz network embedding for inferring novel drug targets.

To predict drug targets, graph-based machine-learning methods have been widely used to capture the relationships between drug, target and disease entities in drug-disease-target (DDT) networks. However, many methods cannot explicitly consider disease types at inference time and so will predict the same target for a given drug under any disease condition. Meanwhile, DDT networks are usually organized hierarchically carrying interactive relationships between involved entities, but these methods, especially those based on Euclidean embedding cannot fully utilize such topological information, which might lead to sub-optimal results. We hypothesized that, by importing hyperbolic embedding specifically for modeling hierarchical DDT networks, graph-based algorithms could better capture relationships between aforementioned entities, which ultimately improves target prediction performance. We formulated the target prediction problem as a knowledge graph completion task explicitly considering disease types. We proposed FLONE, a hyperbolic embedding-based method based on capturing hierarchical topological information in DDT networks. The experimental results on two DDT networks showed that by introducing hyperbolic space, FLONE generates more accurate target predictions than its Euclidean counterparts, which supports our hypothesis. We also devised hyperbolic encoders to fuse external domain knowledge, to make FLONE enable handling samples corresponding to previously unseen drugs and targets for more practical scenarios. Source code and dataset information are at: https://github.com/arantir123/DDT_triple_prediction. Supplementary data are available at Bioinformatics Advances online.

A Neighborhood Re-Ranking Model With Relation Constraint for Knowledge Graph Completion

Knowledge graph completion (KGC) aims to predict missing links based on observed triples. However, current KGC models are still limited by the following two aspects. (1) the entity semantics is implicitly learned by neural network and merely depends on existing facts, which mostly suffers from less additional specific knowledge. Although previous studies have noticed that entity type information can effectively improve KGC task, most of them rely on labeled type-specific data. (2) the recent graph-based models mainly concentrate on Graph Neural Network (GNN) to update source entity representation, regardless of the separate role that neighborhood information plays and may mix noisy neighbor features for target prediction. To address the above two issues, we propose a neighborhood re-ranking model with relation constraint for KGC task. We suggest that both relation constraint and structured information located in triples can boost the model performance. More importantly, we automatically generate explicit constraints as additional type feature to enrich entity representation instead of depending on human annotated labels. Meanwhile, we construct a neighborhood completion module to re-rank candidate entities for full use of the neighbor structure rather than traditional GNN updating manner. Extensive experiments on seven benchmarks demonstrate that our model achieves the competitive results in comparison to the recent advanced baselines.

Boosting BERT-Based Knowledge Graph Completion with Contrastive Learning and Hard Sample Training

Knowledge graph (KG), which is often described by a set of triplets (head, relation, tail), has shown to be very useful for many downstream applications, but suffers from the issue of incomplete connections. Knowledge graph completion (KGC) is to predict the missing connections among entities. Driven by recent advances of BERT in extracting meaningful textual representations, some recent KGC methods have been proposed to leverage the associated texts of triplets to assist the completion. Under the BERT-based framework, we argue that if more meaningful representations can be learned from the texts, better performance can be expected. Following this framework, inspired by recent successes of contrastive learning, we propose to adapt it to the KGC task by introducing a novel dual-encoder architecture to produce semantic-similar positive pairs, which have been widely shown to be pivotal in inducing semantic-rich representations for contrastive learning. To further improve the performance, a hard negative sampling strategy is developed to train the model. Extensive experimental results on three public datasets show that the proposed techniques can improve the performance of existing KGC methods effectively.

Knowledge Graph Completion Based on Entity Descriptions in Hyperbolic Space

Hyperbolic space has received extensive attention because it can accurately and concisely represent hierarchical data. Currently, for knowledge graph completion tasks, the introduction of exogenous information of entities can enrich the knowledge representation of entities, but there is a problem that entities have different levels under different relations, and the embeddings of different entities in Euclidean space often requires high dimensional space to distinguish. Therefore, in order to solve the above problem, we propose a method that use entity descriptions to complete the knowledge graph in the Poincaré ball model of hyperbolic space. In this method, the text representation of entity descriptions is in Euclidean space and mapped to hyperbolic space through exponential map. Next, the triple embeddings are initialized in hyperbolic space, and the structured representation of the triple is trained by the hyperbolic model. Finally, the text representation and the structured representation of the entity are cross-fused in hyperbolic space, and then the balance factors are used to adjust the unbalanced energy function. Experimental results show that, compared with baseline models, the proposed method can improve the performance of knowledge graphs completion.

Knowledge Graph Embedding by Double Limit Scoring Loss

Knowledge graph embedding is an effective way to represent knowledge graph, which greatly enhance the performances on knowledge graph completion tasks, e.g., entity or relation prediction. For knowledge graph embedding models, designing a powerful loss framework is crucial to the discrimination between correct and incorrect triplets. Margin-based ranking loss is a commonly used negative sampling framework to make a suitable margin between the scores of positive and negative triples. However, this loss can not ensure ideal low scores for the positive triplets and high scores for the negative triplets, which is not beneficial for knowledge completion tasks. In this paper, we present a double limit scoring loss to separately set upper bound for correct triplets and lower bound for incorrect triplets, which provides more effective and flexible optimization for knowledge graph embedding. Upon the presented loss framework, we present several knowledge graph embedding models including TransE-SS, TransH-SS, TransD-SS, ProjE-SS and ComplEx-SS. The experimental results on link prediction and triplet classification show that our proposed models have the significant improvement compared to state-of-the-art baselines.