Knowledge Graph Embedding Models Research Articles

A knowledge graph embedding model based on multi-level analogical reasoning

A knowledge graph embedding model based on multi-level analogical reasoning

M[formula omitted]ixKG: Mixing for harder negative samples in knowledge graph

Knowledge graph embedding (KGE) involves mapping entities and relations to low-dimensional dense embeddings, enabling a wide range of real-world applications. The mapping is achieved via distinguishing the positive and negative triplets in knowledge graphs. Therefore, how to design high-quality negative triplets is critical in the effectiveness of KEG models. Existing KGE models face challenges in generating high-quality negative triplets. Some models employ simple static distributions, i.e. uniform or bernoulli distribution, and it is difficult for these methods to be trained distinguishably because of the sampled uninformative negative triplets. Furthermore, current methods are confined to constructing negative triplets from existing entities within the knowledge graph, limiting their ability to explore harder negatives. We introduce a novel mixing strategy in knowledge graphs called M2ixKG. M2ixKG adopts mixing operation in generating harder negative samples from two aspects: one is mixing among the heads and tails in triplets with the same relation to strengthen the robustness and generalization of the entity embeddings; the other is mixing the negatives with high scores to generate harder negatives. Our experiments, utilizing three datasets and four classical score functions, highlight the exceptional performance of M2ixKG in comparison to previous negative sampling algorithms.

Repurposing non-pharmacological interventions for Alzheimer's disease through link prediction on biomedical literature

Non-pharmaceutical interventions (NPI) have great potential to improve cognitive function but limited investigation to discover NPI repurposing for Alzheimer's Disease (AD). This is the first study to develop an innovative framework to extract and represent NPI information from biomedical literature in a knowledge graph (KG), and train link prediction models to repurpose novel NPIs for AD prevention. We constructed a comprehensive KG, called ADInt, by extracting NPI information from biomedical literature. We used the previously-created SuppKG and NPI lexicon to identify NPI entities. Four KG embedding models (i.e., TransE, RotatE, DistMult and ComplEX) and two novel graph convolutional network models (i.e., R-GCN and CompGCN) were trained and compared to learn the representation of ADInt. Models were evaluated and compared on two test sets (time slice and clinical trial ground truth) and the best performing model was used to predict novel NPIs for AD. Discovery patterns were applied to generate mechanistic pathways for high scoring candidates. The ADInt has 162,212 nodes and 1,017,284 edges. R-GCN performed best in time slice (MR = 5.2054, Hits@10 = 0.8496) and clinical trial ground truth (MR = 3.4996, Hits@10 = 0.9192) test sets. After evaluation by domain experts, 10 novel dietary supplements and 10 complementary and integrative health were proposed from the score table calculated by R-GCN. Among proposed novel NPIs, we found plausible mechanistic pathways for photodynamic therapy and Choerospondias axillaris to prevent AD, and validated psychotherapy and manual therapy techniques using real-world data analysis. The proposed framework shows potential for discovering new NPIs for AD prevention and understanding their mechanistic pathways.

Multi-Filter soft shrinkage network for knowledge graph embedding

Incompleteness is a prominent issue pervasive in real-world knowledge graphs, and link prediction techniques, which utilize known facts to forecast missing or unknown links, have emerged as a critical technology for knowledge graph completion. Recently, convolutional neural network-based knowledge graph embedding (KGE) models have achieved breakthroughs in the field of link prediction by increasing the number of interactions between entities and relations. However, current models mainly emphasize stacking complex architectures to capture more interaction features, ignoring the fact that, as the number of features increases significantly, irrelevant features might overshadow important ones, thus impacting the performance of link prediction. To tackle this problem, this paper introduces a novel KGE model that is based on a multi-filter soft shrinkage network (MFSSN). First, this model adaptively constructs filters based on entity embedding and relationship embedding, thereby increasing the interactions between entities and relationships. Secondly, the soft shrinkage function is innovatively introduced into the model, and a specialized soft shrinkage sub-network is innovatively designed to effectively eliminate noise features and improve attention to important features. Finally, the attention mechanism is added to the network to enhance important features and suppress useless features, improving the ability to utilize features, and thereby enhancing model performance. Through extensive experiments on five benchmark datasets of different sizes, the strong performance and generalization ability of the proposed model are proven. Compared with contrasting methods, it achieves performance leadership in almost all indicators.

Fair large kernel embedding with relation-specific features extraction for link prediction

Knowledge graph embedding is a crucial technique for addressing the challenge of incomplete knowledge graphs, and convolutional neural networks are widely applied in this domain. However, convolutional neural networks face limitations in effectively capturing long-range dependencies between entities and relations in feature maps. Moreover, existing knowledge graph embedding models often adopt a uniform perspective on all relations within a knowledge graph, thereby overlooking the relation-specific features. To address the two issues, two novel knowledge graph embedding models, Large Kernel Embedding (LKE) and Large Kernel Embedding with Relation-specific Features Extraction (LKER), are proposed. The effectiveness of knowledge graph embedding is improved by integrating fair large kernel attention and introducing a parallel branch for relation-specific feature extraction in these models. Specifically, fair large kernel attention is incorporated into LKE, capturing long-range dependencies within the model. Based on LKE, LKER adds a parallel relationship-specific feature extraction branch to obtain more comprehensive relationship-specific feature information. The proposed models are extensively evaluated on five benchmark datasets, and the results show that the proposed models yield significant performance improvements in link prediction compared to classical and latest knowledge graph embedding models.

Fast knowledge graph completion using graphics processing units

Knowledge graphs can be used in many areas related to data semantics such as question-answering systems, knowledge based systems. However, the currently constructed knowledge graphs need to be complemented for better knowledge in terms of relations. It is called knowledge graph completion. To add new relations to the existing knowledge graph by using knowledge graph embedding models, we have to evaluate N×N×R vector operations, where N is the number of entities and R is the number of relation types. It is very costly.In this paper, we provide an efficient knowledge graph completion framework on GPUs to get new relations using knowledge graph embedding vectors. In the proposed framework, we first define transformable to a metric space and then provide a method to transform the knowledge graph completion problem into the similarity join problem for a model which is transformable to a metric space. After that, to efficiently process the similarity join problem, we derive formulas using the properties of a metric space. Based on the formulas, we develop a fast knowledge graph completion algorithm. Finally, we experimentally show that our framework can efficiently process the knowledge graph completion problem.

Towards Continual Knowledge Graph Embedding via Incremental Distillation

Traditional knowledge graph embedding (KGE) methods typically require preserving the entire knowledge graph (KG) with significant training costs when new knowledge emerges. To address this issue, the continual knowledge graph embedding (CKGE) task has been proposed to train the KGE model by learning emerging knowledge efficiently while simultaneously preserving decent old knowledge. However, the explicit graph structure in KGs, which is critical for the above goal, has been heavily ignored by existing CKGE methods. On the one hand, existing methods usually learn new triples in a random order, destroying the inner structure of new KGs. On the other hand, old triples are preserved with equal priority, failing to alleviate catastrophic forgetting effectively. In this paper, we propose a competitive method for CKGE based on incremental distillation (IncDE), which considers the full use of the explicit graph structure in KGs. First, to optimize the learning order, we introduce a hierarchical strategy, ranking new triples for layer-by-layer learning. By employing the inter- and intra-hierarchical orders together, new triples are grouped into layers based on the graph structure features. Secondly, to preserve the old knowledge effectively, we devise a novel incremental distillation mechanism, which facilitates the seamless transfer of entity representations from the previous layer to the next one, promoting old knowledge preservation. Finally, we adopt a two-stage training paradigm to avoid the over-corruption of old knowledge influenced by under-trained new knowledge. Experimental results demonstrate the superiority of IncDE over state-of-the-art baselines. Notably, the incremental distillation mechanism contributes to improvements of 0.2%-6.5% in the mean reciprocal rank (MRR) score. More exploratory experiments validate the effectiveness of IncDE in proficiently learning new knowledge while preserving old knowledge across all time steps.

Knowledge Graph Embedding: A Survey from the Perspective of Representation Spaces

Knowledge graph embedding (KGE) is an increasingly popular technique that aims to represent entities and relations of knowledge graphs into low-dimensional semantic spaces for a wide spectrum of applications such as link prediction, knowledge reasoning and knowledge completion. In this article, we provide a systematic review of existing KGE techniques based on representation spaces. Particularly, we build a fine-grained classification to categorise the models based on three mathematical perspectives of the representation spaces: (1) algebraic perspective, (2) geometric perspective and (3) analytical perspective. We introduce the rigorous definitions of fundamental mathematical spaces before diving into KGE models and their mathematical properties. We further discuss different KGE methods over the three categories, as well as summarise how spatial advantages work over different embedding needs. By collating the experimental results from downstream tasks, we also explore the advantages of mathematical space in different scenarios and the reasons behind them. We further state some promising research directions from a representation space perspective, with which we hope to inspire researchers to design their KGE models as well as their related applications with more consideration of their mathematical space properties.

Multi-perspective knowledge graph completion with global and interaction features

Knowledge graphs are multi-relation heterogeneous graphs. Thus, the existence of numerous multi-relation entities imposes a tough challenge to the modelling of the knowledge graph. Some recent works represent the property of corresponding entities and relations by generating embeddings. They attempted to identify the missing entities by translation operations or semantic matching. However, the expressiveness of these approaches depends on the entity (relations) embedding. The heterogeneity of entities leads to the difficulty of balancing uniform embedding dimension settings on complex and sparse relational entities, as high-dimensional embedding leads to the overfitting of sparse relational entities, and low-dimensional embedding leads to the underfitting of complex relational entities. We introduce a multi-perspective knowledge graph embedding model with global and interaction features (MGIF) to alleviate these issues. This achieved knowledge transfer from complex relational entities to sparse relational entities through the multi-view features. In particular, to overcome the local limitations of convolution neural networks, the global features shared between entities (relations) and entities (relations) are incorporated in the MGIF. The performance of MGIF is experimentally evaluated on several datasets. The experimental effects demonstrate that MGIF can efficiently model complicated entities and accomplish state-of-the-art complex relationship prediction results on most evaluation metrics.

Drug–target interaction prediction using knowledge graph embedding

Drug–target interaction prediction using knowledge graph embedding

A collaborative learning framework for knowledge graph embedding and reasoning

Knowledge graph embedding (KGE) and knowledge graph reasoning (KGR) aim to automatic completion of knowledge graph (KG). The difference is that most KGE models learn the embedded representation at a triple level. In contrast, KGR models focus more on optimizing decision-making and enhancing the interpretability of reasoning processes with multi-hop paths. As a result, KGE models are better at learning triplet embeddings, whereas KGR models can capture the multihop information between entity pairs. However, KGE and KGR models only focus on one aspect that affects the completion performance. This paper proposes a plug-and-play collaborative learning framework (CLF) for jointly enhancing knowledge graph embedding and reasoning, which can accommodate existing KGR and KGE models. The two models exchange training experiences in this framework to realize mutual learning through a collaborative learning module. In this module, a new distance function is designed to maintain the independence of candidate entities’ probabilities and avoid information loss. Furthermore, a knowledge augmentation module is designed to identify missing key triples to assist in the further iterative training of the framework. Extensive experiments on the benchmark datasets demonstrate that our framework significantly improves the performance of existing models.

Knowledge Graph Embedding via Triplet Component Interactions

In knowledge graph embedding, multidimensional representations of entities and relations are learned in vector space. Although distance-based graph embedding methods have shown promise in link prediction, they neglect context information among the triplet components, i.e., the head_entity, relation, and tail_entity, limiting their ability to describe multivariate relation patterns and mapping properties. Such context information denotes the entity structural association inside the same triplet and implies the correlation between entities that are not directly connected. In this work, we propose a novel knowledge graph embedding model that explicitly considers context information in graph embedding via triplet component interactions (TCIE). To build connections between components and incorporate contextual information, entities and relations are represented as vectors comprised of two specialized parts, enabling comprehensive interaction. By simultaneously interacting with one-hop related head and tail entities, TCIE strengthens the connections between distant entities and enables contextual information to be transmitted across the knowledge graph. Mathematical proofs and experiments are performed to analyse the modelling ability of TCIE in knowledge graph embedding. TCIE shows a strong capacity for modelling four relation patterns (i.e., symmetry, antisymmetry, inverse, and composition) and four mapping properties (i.e., one-to-one, one-to-many, many-to-one, and many-to-many). The experimental evaluation of ogbl-wikikg2, ogbl-biokg, FB15k, and FB15k-237 shows that TCIE achieves state-of-the-art results in link prediction.

Knowledge graph embedding in a uniform space

Knowledge graph embedding (KGE) is typically used for link prediction to automatically predict missing links in knowledge graphs. Current KGE models are mainly based on complicated mathematical associations, which are highly expressive but ignore the uniformity behind the classical bilinear translational model TransE, a model that embeds all entities of knowledge graphs in a uniform space, enabling accurate embeddings. This study analyses the uniformity of TransE and proposes a novel KGE model called ConvUs that follows uniformity with expressiveness. Based on the convolution neural network (CNN), ConvUs proposes constraints on convolution filter values and employs a multi-layer, multi-scale CNN architecture with a non-parametric L2 norm-based scoring function for the calculation of triple scores. This addresses potential uniformity-related issues in existing CNN-based KGE models, allowing ConvUs to maintain a uniform embedding space while benefiting from the powerful expressiveness of CNNs. Furthermore, circular convolution is applied to alleviate the potential orderliness contradictions, making ConvUs more suitable for conducting uniform space KGE. Our model outperformed the base model ConvKB and several baselines on the link prediction benchmark WN18RR and FB15k-237, demonstrating strong applicability and generalization and indicating that the uniformity of embedding space with high expressiveness enables more efficient knowledge graph embeddings.

Semantic-Enhanced Knowledge Graph Completion

Knowledge graphs (KGs) serve as structured representations of knowledge, comprising entities and relations. KGs are inherently incomplete, sparse, and have a strong need for completion. Although many knowledge graph embedding models have been designed for knowledge graph completion, they predominantly focus on capturing observable correlations between entities. Due to the sparsity of KGs, potential semantic correlations are challenging to capture. To tackle this problem, we propose a model entitled semantic-enhanced knowledge graph completion (SE-KGC). SE-KGC effectively addresses the issue by incorporating predefined semantic patterns, enabling the capture of semantic correlations between entities and enhancing features for representation learning. To implement this approach, we employ a multi-relational graph convolution network encoder, which effectively encodes the KG. Subsequently, we utilize a scoring decoder to evaluate triplets. Experimental results demonstrate that our SE-KGC model outperforms other state-of-the-art methods in link-prediction tasks across three datasets. Specifically, compared to the baselines, SE-KGC achieved improvements of 11.7%, 1.05%, and 2.30% in terms of MRR on these three datasets. Furthermore, we present a comprehensive analysis of the contributions of different semantic patterns, and find that entities with higher connectivity play a pivotal role in effectively capturing and characterizing semantic information.

A knowledge graph embedding model based attention mechanism for enhanced node information integration.

The purpose of knowledge embedding is to extract entities and relations from the knowledge graph into low-dimensional dense vectors, in order to be applied to downstream tasks, such as connection prediction and intelligent classification. Existing knowledge embedding methods still have many limitations, such as the contradiction between the vast amount of data and limited computing power, and the challenge of effectively representing rare entities. This article proposed a knowledge embedding learning model, which incorporates a graph attention mechanism to integrate key node information. It can effectively aggregate key information from the global graph structure, shield redundant information, and represent rare nodes in the knowledge base independently of its own structure. We introduce a relation update layer to further update the relation based on the results of entity training. The experiment shows that our method matches or surpasses the performance of other baseline models in link prediction on the FB15K-237 dataset. The metric Hits@1 has increased by 10.9% compared to the second-ranked baseline model. In addition, we conducted further analysis on rare nodes with fewer neighborhoods, confirming that our model can embed rare nodes more accurately than the baseline models.

Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure

Knowledge graph embedding (KGE) aims at learning powerful representations to benefit various artificial intelligence applications. Meanwhile, contrastive learning has been widely leveraged in graph learning as an effective mechanism to enhance the discriminative capacity of the learned representations. However, the complex structures of KG make it hard to construct appropriate contrastive pairs. Only a few attempts have integrated contrastive learning strategies with KGE. But, most of them rely on language models ( e.g., Bert) for contrastive pair construction instead of fully mining information underlying the graph structure, hindering expressive ability. Surprisingly, we find that the entities within a relational symmetrical structure are usually similar and correlated. To this end, we propose a knowledge graph contrastive learning framework based on relation-symmetrical structure, KGE-SymCL, which mines symmetrical structure information in KGs to enhance the discriminative ability of KGE models. Concretely, a plug-and-play approach is proposed by taking entities in the relation-symmetrical positions as positive pairs. Besides, a self-supervised alignment loss is designed to pull together positive pairs. Experimental results on link prediction and entity classification datasets demonstrate that our KGE-SymCL can be easily adopted to various KGE models for performance improvements. Moreover, extensive experiments show that our model could outperform other state-of-the-art baselines.

A semantic Enhanced Knowledge Graph Embedding Model With AIGC Designed for Healthcare Prediction

A semantic Enhanced Knowledge Graph Embedding Model With AIGC Designed for Healthcare Prediction

Improving Landslide Prediction: Innovative Modeling and Evaluation of Landslide Scenario with Knowledge Graph Embedding

The increasing frequency and magnitude of landslides underscore the growing importance of landslide prediction in light of factors like climate change. Traditional methods, including physics-based methods and empirical methods, are beset by high costs and a reliance on expert knowledge. With the advancement of remote sensing and machine learning, data-driven methods have emerged as the mainstream in landslide prediction. Despite their strong generalization capabilities and efficiency, data-driven methods suffer from the loss of semantic information during training due to their reliance on a ‘sequence’ modeling method for landslide scenarios, which impacts their predictive accuracy. An innovative method for landslide prediction is proposed in this paper. In this paper, we propose an innovative landslide prediction method. This method designs the NADE ontology as the schema layer and constructs the data layer of the knowledge graph, utilizing tile lists, landslide inventory, and environmental data to enhance the representation of complex landslide scenarios. Furthermore, the transformation of the landslide prediction task into a link prediction task is carried out, and a knowledge graph embedding model is trained to achieve landslide predictions. Experimental results demonstrate that the method improves the F1 score by 5% in scenarios with complete datasets and 17% in scenarios with sparse datasets compared to data-driven methods. Additionally, the application of the knowledge graph embedding model is utilized to generate susceptibility maps, and an analysis of the effectiveness of entity embeddings is conducted, highlighting the potential of knowledge graph embeddings in disaster management.

Sem@K: Is my knowledge graph embedding model semantic-aware?

Using knowledge graph embedding models (KGEMs) is a popular approach for predicting links in knowledge graphs (KGs). Traditionally, the performance of KGEMs for link prediction is assessed using rank-based metrics, which evaluate their ability to give high scores to ground-truth entities. However, the literature claims that the KGEM evaluation procedure would benefit from adding supplementary dimensions to assess. That is why, in this paper, we extend our previously introduced metric Sem@K that measures the capability of models to predict valid entities w.r.t. domain and range constraints. In particular, we consider a broad range of KGs and take their respective characteristics into account to propose different versions of Sem@K. We also perform an extensive study to qualify the abilities of KGEMs as measured by our metric. Our experiments show that Sem@K provides a new perspective on KGEM quality. Its joint analysis with rank-based metrics offers different conclusions on the predictive power of models. Regarding Sem@K, some KGEMs are inherently better than others, but this semantic superiority is not indicative of their performance w.r.t. rank-based metrics. In this work, we generalize conclusions about the relative performance of KGEMs w.r.t. rank-based and semantic-oriented metrics at the level of families of models. The joint analysis of the aforementioned metrics gives more insight into the peculiarities of each model. This work paves the way for a more comprehensive evaluation of KGEM adequacy for specific downstream tasks.

A Knowledge Graph Embedding Model Based on Cyclic Consistency—Cyclic_CKGE

Most of the existing medical knowledge maps are incomplete and need to be completed/predicted to obtain a complete knowledge map. To solve this problem, we propose a knowledge graph embedding model (Cyclic_CKGE) based on cyclic consistency. The model first uses the “graph” constructed with the head entity and relationship to predict the tail entity, and then uses the “inverse graph” constructed with the tail entity and relationship to predict the head entity. Finally, the semantic space distance between the head entity and the original head entity should be very close, which solves the reversibility problem of the network. The Cyclic_CKGE model with a parameter of 0.46 M has the best results on FB15k-237, reaching 0.425 Hits@10. Compared with the best model R-GCN, its parameter exceeds 8 M and reaches 0.417 Hits@10. Overall, Cyclic_CKGE’s parametric efficiency is more than 17 times that of R-GCNs and more than 8 times that of DistMult. In order to better show the practical application of the model, we construct a visual medical information platform based on a medical knowledge map. The platform has three kinds of disease information retrieval methods: conditional query, path query and multi-symptom disease inference. This provides a theoretical method and a practical example for realizing knowledge graph visualization.