Knowledge Graph Completion Models Research Articles

An experimentally validated approach to automated biological evidence generation in drug discovery using knowledge graphs

Explaining predictions for drug repositioning with biological knowledge graphs is a challenging problem. Graph completion methods using symbolic reasoning predict drug treatments and associated rules to generate evidence representing the therapeutic basis of the drug. Yet the vast amounts of generated paths that are biologically irrelevant or not mechanistically meaningful within the context of disease biology can limit utility. We use a reinforcement learning based knowledge graph completion model combined with an automatic filtering approach that produces the most relevant rules and biological paths explaining the predicted drug’s therapeutic connection to the disease. In this work we validate the approach against preclinical experimental data for Fragile X syndrome demonstrating strong correlation between automatically extracted paths and experimentally derived transcriptional changes of selected genes and pathways of drug predictions Sulindac and Ibudilast. Additionally, we show it reduces the number of generated paths in two case studies, 85% for Cystic fibrosis and 95% for Parkinson’s disease.

Enhancing text-based knowledge graph completion with zero-shot large language models: A focus on semantic enhancement

The design and development of text-based knowledge graph completion (KGC) methods leveraging textual entity descriptions are at the forefront of research. These methods involve advanced optimization techniques such as soft prompts and contrastive learning to enhance KGC models. The effectiveness of text-based methods largely hinges on the quality and richness of the training data. Large language models (LLMs) can utilize straightforward prompts to alter text data, thereby enabling data augmentation for KGC. Nevertheless, LLMs typically demand substantial computational resources. To address these issues, we introduce a framework termed constrained prompts for KGC (CP-KGC). This CP-KGC framework designs prompts that adapt to different datasets to enhance semantic richness. Additionally, CP-KGC employs a context constraint strategy to effectively identify polysemous entities within KGC datasets. Through extensive experimentation, we have verified the effectiveness of this framework. Even after quantization, the LLM (Qwen-7B-Chat-int4) still enhances the performance of text-based KGC methods.11Code and datasets are available at https://github.com/sjlmg/CP-KGC. This study extends the performance limits of existing models and promotes further integration of KGC with LLMs.

Knowledge graph completion model based on hyperbolic hierarchical attention network

Knowledge graph completion model based on hyperbolic hierarchical attention network

GLSEC: Global and local semantic-enhanced contrastive framework for knowledge graph completion

Knowledge graph completion (KGC) aims to infer missing links between entities in knowledge graphs (KGs). Recently, models based on graph neural networks (GNNs) have gained widespread attention due to their effectiveness in leveraging the topological structure information of entities. Meanwhile, contrastive learning (CL) has been employed in GNNs-based models to provide more supervised signals for better entity representation in a self-supervised manner. However, existing methods overlook the potential global semantic collaboration among entities within the entire KG. And the application of CL in KGC models often adopt random graph augmentation or basic node structure contrast, leading to suboptimal performance. To tackle them, we propose a Global and Local Semantic-Enhanced Contrastive Framework (GLSEC) for KGC. Specifically, we develop a global Attribute-aware encoder to capture the global semantic features of entities based on an entity-entity Attribute Interaction Graph (AIG). Additionally, we design a Light Graph Aggregation Network (Light-GAN) that innovatively updates the global semantic features using the AIG, combining both efficiency and a lightweight design. Furthermore, we introduce a Global-Local cross-view Contrastive Learning (GLCL) method that contrasts embeddings from global and local views, thereby improving contrastive sample quality and boosting the model’s understanding of entities in various contexts. Extensive experiments show that our model outperforms state-of-the-art KGC methods on benchmark datasets FB15k-237 and WN18RR.

Mixed Geometry Message and Trainable Convolutional Attention Network for Knowledge Graph Completion

Knowledge graph completion (KGC) aims to study the embedding representation to solve the incompleteness of knowledge graphs (KGs). Recently, graph convolutional networks (GCNs) and graph attention networks (GATs) have been widely used in KGC tasks by capturing neighbor information of entities. However, Both GCNs and GATs based KGC models have their limitations, and the best method is to analyze the neighbors of each entity (pre-validating), while this process is prohibitively expensive. Furthermore, the representation quality of the embeddings can affect the aggregation of neighbor information (message passing). To address the above limitations, we propose a novel knowledge graph completion model with mixed geometry message and trainable convolutional attention network named MGTCA. Concretely, the mixed geometry message function generates rich neighbor message by integrating spatially information in the hyperbolic space, hypersphere space and Euclidean space jointly. To complete the autonomous switching of graph neural networks (GNNs) and eliminate the necessity of pre-validating the local structure of KGs, a trainable convolutional attention network is proposed by comprising three types of GNNs in one trainable formulation. Furthermore, a mixed geometry scoring function is proposed, which calculates scores of triples by novel prediction function and similarity function based on different geometric spaces. Extensive experiments on three standard datasets confirm the effectiveness of our innovations, and the performance of MGTCA is significantly improved compared to the state-of-the-art approaches.

Modeling Knowledge Graphs with Composite Reasoning

The ability to combine multiple pieces of existing knowledge to infer new knowledge is both crucial and challenging. In this paper, we explore how facts of various entities are combined in the context of knowledge graph completion (KGC). We use composite reasoning to unify the views from different KGC models, including translational models, tensor factorization (TF)-based models, instance-based learning models, and KGC regularizers. Moreover, our comprehensive examination of composite reasoning revealed an unexpected phenomenon: certain TF-based models learn embeddings with erroneous composite reasoning, which ultimately violates their fundamental collaborative filtering assumption and reduces their effects. This motivates us to reduce their composition error. Empirical evaluations demonstrate that mitigating the composition risk not only enhances the performance of TF-based models across all tested settings, but also surpass or is competitive with the state-of-the-art performance on two out of four benchmarks.

LAFA: Multimodal Knowledge Graph Completion with Link Aware Fusion and Aggregation

Recently, an enormous amount of research has emerged on multimodal knowledge graph completion (MKGC), which seeks to extract knowledge from multimodal data and predict the most plausible missing facts to complete a given multimodal knowledge graph (MKG). However, existing MKGC approaches largely ignore that visual information may introduce noise and lead to uncertainty when adding them to the traditional KG embeddings due to the contribution of each associated image to entity is different in diverse link scenarios. Moreover, treating each triple independently when learning entity embeddings leads to local structural and the whole graph information missing. To address these challenges, we propose a novel link aware fusion and aggregation based multimodal knowledge graph completion model named LAFA, which is composed of link aware fusion module and link aware aggregation module. The link aware fusion module alleviates noise of irrelevant visual information by calculating the importance between an entity and its associated images in different link scenarios, and fuses the visual and structural embeddings according to the importance through our proposed modality embedding fusion mechanism. The link aware aggregation module assigns neighbor structural information to a given central entity by calculating the importance between the entity and its neighbors, and aggregating the fused embeddings through linear combination according to the importance. Extensive experiments on standard datasets validate that LAFA can obtain state-of-the-art performance.

Simple knowledge graph completion model based on PU learning and prompt learning

Simple knowledge graph completion model based on PU learning and prompt learning

A Visualization Method of Knowledge Graphs for the Computation and Comprehension of Ultrasound Reports.

Knowledge graph visualization in ultrasound reports is essential for enhancing medical decision making and the efficiency and accuracy of computer-aided analysis tools. This study aims to propose an intelligent method for analyzing ultrasound reports through knowledge graph visualization. Firstly, we provide a novel method for extracting key term networks from the narrative text in ultrasound reports with high accuracy, enabling the identification and annotation of clinical concepts within the report. Secondly, a knowledge representation framework based on ultrasound reports is proposed, which enables the structured and intuitive visualization of ultrasound report knowledge. Finally, we propose a knowledge graph completion model to address the lack of entities in physicians' writing habits and improve the accuracy of visualizing ultrasound knowledge. In comparison to traditional methods, our proposed approach outperforms the extraction of knowledge from complex ultrasound reports, achieving a significantly higher extraction index (η) of 2.69, surpassing the general pattern-matching method (2.12). In comparison to other state-of-the-art methods, our approach achieves the highest P (0.85), R (0.89), and F1 (0.87) across three testing datasets. The proposed method can effectively utilize the knowledge embedded in ultrasound reports to obtain relevant clinical information and improve the accuracy of using ultrasound knowledge.

Enhanced Temporal Knowledge Graph Completion via Learning High-Order Connectivity and Attribute Information

Temporal knowledge graph completion (TKGC) refers to the prediction and filling in of missing facts on time series, which is essential for many downstream applications. However, many existing TKGC methods suffer from two limitations: (1) they only consider direct relations between entities and fail to express high-order structural dependencies between entities; and (2) they only leverage relation quadruples of temporal knowledge graphs, ignoring attribute information that contains rich semantic information. This makes them vulnerable to sparsity and incompleteness problems. In response, we propose HCAE, a temporal knowledge graph completion model that includes high-order connectivity and attribute information. This consists mainly of a recursive embedding propagation layer and a multi-head attention aggregation layer. The former leverages a recursive mechanism to update entity embeddings and can learn high-order connectivity information between entities in linear complexity time. The latter leverages an attention mechanism to understand the importance of different attributes for entity representation automatically. Combining high-order connectivity and attribute information can lead to more diverse entity representations and help enhance the model’s ability to infer unknown entities. Comparative experiments on three real-world datasets show that the model’s inference accuracy significantly outperforms other benchmark methods, especially regarding knowledge graphs with many unknown entities or relations.

Knowledge graph completion method based on hyperbolic representation learning and contrastive learning

Knowledge graph completion employs existing triples to deduce missing data, thereby enriching and enhancing graph completeness. Recent research has revealed that using hyperbolic representation learning in knowledge graph completion yields superior expressive and generalization capabilities. However, the long-tail problem and the presence of hyperbolic metrics make it challenging to effectively learn low-frequency entities or relations, resulting in embedding space distortion and impacting the original semantic relationships. Therefore, this paper proposes a knowledge graph completion method (Att-CL) that integrates hyperbolic representation learning and contrastive learning. In this approach, knowledge is embedded into a hyperbolic space, and samples with limited hierarchical characteristics and insufficient feature information are enhanced by introducing adversarial noise. The loss function of the embedded samples is backpropagated into embedding vectors, perturbations are adjusted in the gradient direction to promote smoothness and locality, and hyperparameters are introduced for fine-tuning the adversarial strength in the construction of adversarial samples for data augmentation to enhance model robustness. To mitigate data distortion due to hyperbolic metrics, a penalty term is introduced in the contrastive loss function to control the distances of the embedding vectors from the origin, thereby reducing the impact of the metrics and further improving the model's completion ability. Experimental results on the WN18RR and FB15K-237 benchmark datasets demonstrate significant improvements in metrics such as MRR, Hits@1, and Hits@3 compared to traditional knowledge graph completion models, providing ample evidence of the model's effectiveness.

Few-Shot Knowledge Graph Completion Model Based on Relation Learning

Considering the complexity of entity pair relations and the information contained in the target neighborhood in few-shot knowledge graphs (KG), existing few-shot KG completion methods generally suffer from insufficient relation representation learning capabilities and neglecting the contextual semantics of entities. To tackle the above problems, we propose a Few-shot Relation Learning-based Knowledge Graph Completion model (FRL-KGC). First, a gating mechanism is introduced during the aggregation of higher-order neighborhoods of entities in formation, enriching the central entity representation while reducing the adverse effects of noisy neighbors. Second, during the relation representation learning stage, a more accurate relation representation is learned by using the correlation between entity pairs in the reference set. Finally, an LSTM structure is incorporated into the Transformer learner to enhance its ability to learn the contextual semantics of entities and relations and predict new factual knowledge. We conducted comparative experiments on the publicly available NELL-One and Wiki-One datasets, comparing FRL-KGC with six few-shot knowledge graph completion models and five traditional knowledge graph completion models for five-shot link prediction. The results showed that FRL-KGC outperformed all comparison models in terms of MRR, Hits@10, Hits@5, and Hits@1 metrics.

Simple Knowledge Graph Completion Model Based on Differential Negative Sampling and Prompt Learning

Knowledge graphs (KGs) serve as a crucial resource for numerous artificial intelligence tasks, significantly contributing to the advancement of the AI field. However, the incompleteness of existing KGs hinders their effectiveness in practical applications. Consequently, researchers have proposed the task of KG completion. Currently, embedding-based techniques dominate the field as they leverage the structural information within KGs to infer and complete missing parts. Nonetheless, these methods exhibit limitations. They are limited by the quality and quantity of structural information and are unable to handle the missing entities in the original KG. To overcome these challenges, researchers have attempted to integrate pretrained language models and textual data to perform KG completion. This approach utilizes the definition statements and description text of entities within KGs. The goal is to compensate for the latent connections that are difficult for traditional methods to obtain. However, text-based methods still lag behind embedding-based models in terms of performance. Our analysis reveals that the critical issue lies in the selection process of negative samples. In order to enhance the performance of the text-based methods, various types of negative sampling methods are employed in this study. We introduced prompt learning to fill the gap between the pre-training language model and the knowledge graph completion task, and to improve the model reasoning level. Simultaneously, a ranking strategy based on KG structural information is proposed to utilize KG structured data to assist reasoning. The experiment results demonstrate that our model exhibits strong competitiveness and outstanding inference speed. By fully exploiting the internal structural information of KGs and external relevant descriptive text resources, we successfully elevate the performance levels of KG completion tasks across various metrics.

SFCA: A Scalable Formal Concepts Driven Architecture for Multi-Field Knowledge Graph Completion

With the proliferation of Knowledge Graphs (KGs), knowledge graph completion (KGC) has attracted much attention. Previous KGC methods focus on extracting shallow structural information from KGs or in combination with external knowledge, especially in commonsense concepts (generally, commonsense concepts refer to the basic concepts in related fields that are required for various tasks and academic research, for example, in the general domain, “Country” can be considered as a commonsense concept owned by “China”), to predict missing links. However, the technology of extracting commonsense concepts from the limited database is immature, and the scarce commonsense database is also bound to specific verticals (commonsense concepts vary greatly across verticals, verticals refer to a small field subdivided vertically under a large field). Furthermore, most existing KGC models refine performance on public KGs, leading to inapplicability to actual KGs. To address these limitations, we proposed a novel Scalable Formal Concept-driven Architecture (SFCA) to automatically encode factual triples into formal concepts as a superior structural feature, to support rich information to KGE. Specifically, we generate dense formal concepts first, then yield a handful of entity-related formal concepts by sampling and delimiting the appropriate candidate entity range via the filtered formal concepts to improve the inference of KGC. Compared with commonsense concepts, KGC benefits from more valuable information from the formal concepts, and our self-supervision extraction method can be applied to any KGs. Comprehensive experiments on five public datasets demonstrate the effectiveness and scalability of SFCA. Besides, the proposed architecture also achieves the SOTA performance on the industry dataset. This method provides a new idea in the promotion and application of knowledge graphs in AI downstream tasks in general and industrial fields.

RoAN: A relation-oriented attention network for temporal knowledge graph completion

In the last few years, the availability of temporal knowledge graphs (TKGs), which associate time information for each event, increased the need for completing in these TKGs. In order to solve the completion problem, the existing efforts extend static knowledge graph completion models to handle time-dependent representations, or encode the structural information of events for passing temporal message. However, the above efforts mostly construct TKGs in the form of multi-edges mesh and focus on entity features. They do not explore the power of relations for TKG completion task. In this paper, we introduce a form of relational multi-chains to reconstruct TKGs and propose a relation-oriented attention mechanism for embedding the features of relations. According to our attention mechanism, we build a Relation-oriented Attention Network (RoAN) to model temporal embeddings of relations. It is worth noting that our approach is model-agnostic and can be potentially combined with most existing TKG completion models. Experimental results show that our approach can be coupled with previous TKGC methods and can increase their performance accordingly. In addition, the analysis reveals the principle of our relation-oriented attention mechanism.

A Cybersecurity Knowledge Graph Completion Method for Penetration Testing

Penetration testing is an effective method of making computers secure. When conducting penetration testing, it is necessary to fully understand the various elements in the cyberspace. Prediction of future cyberspace state through perception and understanding of cyberspace can assist defenders in decision-making and action execution. Accurate cyberspace detection information is the key to ensuring successful penetration testing. However, cyberspace situation awareness still faces the following challenges. Due to the limited detection capability, the information obtained from cyberspace detection intelligence is incomplete. There are some errors in the cyberspace detection intelligence, which may mislead the penetration testing workers. The knowledge graph can store and manage the cybersecurity data. In order to ensure the integrity and accuracy of cyberspace information, we design a knowledge graph completion model called CSNT to complete cybersecurity data. CSNT uses the BiLSTM to capture the interaction information between entities and relationships. It models the relationship between entities by combining the neural network and tensor decomposition. The Pearson Mix Net is designed to control the generation of joint vectors. We also design a novel self-distillation strategy to reduce catastrophic forgetting during model training. After learning the relationship pattern between entities in the cyberspace detection intelligence, the model can be used to mine the knowledge not found in the cybersecurity detection intelligence and correct the erroneous records. Experiments show that our method has certain advantages for the knowledge graph completion.

ConvMS: Improving Convolutional Knowledge Graph Embeddings via Integrating Information of Multiple Subspaces

Knowledge graphs are involved in more and more applications to further improve intelligence. Owing to the inherent incompleteness of knowledge graphs resulted from data updating and missing, a number of knowledge graph completion models are proposed in succession. To obtain better performance, many methods are of high complexity, making it time-consuming for training and inference. This paper proposes a simple but effective model using only shallow neural networks, which combines enhanced feature interaction and multi-subspace information integration. In the enhanced feature interaction module, entity and relation embeddings are almost peer-to-peer interacted via multi-channel 2D convolution. In the multi-subspace information integration module, entity and relation embeddings are projected to multiple subspaces to extract multi-view information to further boost performance. Extensive experiments on widely used datasets show that the proposed model outperforms a series of strong baselines. And ablation studies demonstrate the effectiveness of each submodule in the model.

Knowledge graph extension with a pre-trained language model via unified learning method

Knowledge graphs (KGs) are collections of real-world knowledge that is represented by a structured form of triples. Since they are manually built in their nascent stage, there is a common problem that some links (triples) are missing. Knowledge graph completion (KGC) aims to find those missing links and thereby complete the KGs. However, as knowledge increases through diverse sources, new entities have explosively emerged and they are needed to be connected to existing KGs. Thus, open-world KGC is targeted on extending KGs to those new entities. Dealing with those new entities is challenging because they do not have any connection with entities in the existing KGs. One way to handle the new ones is to embed them with their textual descriptions with pre-trained word embeddings and score them in the graph-vector space with the existing typical KGC models. These models have resulted in meaningful results but there is still a lack of studies on utilizing the latest neural networks, such as pre-trained language models which are known to be better at capturing contexts than pre-trained word embeddings. This paper proposes a novel model that effectively connects new entities and existing KGs through a pre-trained language model. To effectively handle the problem, we utilize two learning methods; one is the classification method of the masked language model (MLM) that predicts a word among a huge vocabulary set with a given context, and the other is multi-task learning based on the Multi-Task for Deep Neural Networks (MT-DNN). Based on the methods, the model first generates an embedding of a new entity using its textual description and then uses the embedding to find one of the existing entities from a KG where the new entity can be connected. The experimental results on three benchmark datasets, DBPedia50k, FB15k-237-OWE, and FB20k, show that the proposed model improves performances by 9.2%p, 4.4%p, and 11.1%p, respectively, and achieves new state-of-the-art performance for all datasets.

MRE: A translational knowledge graph completion model based on multiple relation embedding.

Knowledge graph completion (KGC) has attracted significant research interest in applying knowledge graphs (KGs). Previously, many works have been proposed to solve the KGC problem, such as a series of translational and semantic matching models. However, most previous methods suffer from two limitations. First, current models only consider the single form of relations, thus failing to simultaneously capture the semantics of multiple relations (direct, multi-hop and rule-based). Second, the data-sparse problem of knowledge graphs would make part of relations challenging to embed. This paper proposes a novel translational knowledge graph completion model named multiple relation embedding (MRE) to address the above limitations. We attempt to embed multiple relations to provide more semantic information for representing KGs. To be more specific, we first leverage PTransE and AMIE+ to extract multi-hop and rule-based relations. Then, we propose two specific encoders to encode extracted relations and capture semantic information of multiple relations. We note that our proposed encoders can achieve interactions between relations and connected entities in relation encoding, which is rarely considered in existing methods. Next, we define three energy functions to model KGs based on the translational assumption. At last, a joint training method is adopted to perform KGC. Experimental results illustrate that MRE outperforms other baselines on KGC, demonstrating the effectiveness of embedding multiple relations for advancing knowledge graph completion.

A Neighborhood Re-Ranking Model With Relation Constraint for Knowledge Graph Completion

Knowledge graph completion (KGC) aims to predict missing links based on observed triples. However, current KGC models are still limited by the following two aspects. (1) the entity semantics is implicitly learned by neural network and merely depends on existing facts, which mostly suffers from less additional specific knowledge. Although previous studies have noticed that entity type information can effectively improve KGC task, most of them rely on labeled type-specific data. (2) the recent graph-based models mainly concentrate on Graph Neural Network (GNN) to update source entity representation, regardless of the separate role that neighborhood information plays and may mix noisy neighbor features for target prediction. To address the above two issues, we propose a neighborhood re-ranking model with relation constraint for KGC task. We suggest that both relation constraint and structured information located in triples can boost the model performance. More importantly, we automatically generate explicit constraints as additional type feature to enrich entity representation instead of depending on human annotated labels. Meanwhile, we construct a neighborhood completion module to re-rank candidate entities for full use of the neighbor structure rather than traditional GNN updating manner. Extensive experiments on seven benchmarks demonstrate that our model achieves the competitive results in comparison to the recent advanced baselines.