Relations In Knowledge Graph Research Articles

Multi-relational graph contrastive learning with learnable graph augmentation

Multi-relational graph learning aims to embed entities and relations in knowledge graphs into low-dimensional representations, which has been successfully applied to various multi-relationship prediction tasks, such as information retrieval, question answering, and etc. Recently, contrastive learning has shown remarkable performance in multi-relational graph learning by data augmentation mechanisms to deal with highly sparse data. In this paper, we present a Multi-Relational Graph Contrastive Learning architecture (MRGCL) for multi-relational graph learning. More specifically, our MRGCL first proposes a Multi-relational Graph Hierarchical Attention Networks (MGHAN) to identify the importance between entities, which can learn the importance at different levels between entities for extracting the local graph dependency. Then, two graph augmented views with adaptive topology are automatically learned by the variant MGHAN, which can automatically adapt for different multi-relational graph datasets from diverse domains. Moreover, a subgraph contrastive loss is designed, which generates positives per anchor by calculating strongly connected subgraph embeddings of the anchor as the supervised signals. Comprehensive experiments on multi-relational datasets from three application domains indicate the superiority of our MRGCL over various state-of-the-art methods. Our datasets and source code are published at https://github.com/Legendary-L/MRGCL.

C-KGE: Curriculum learning-based Knowledge Graph Embedding

Knowledge graph embedding (KGE) aims to embed entities and relations in knowledge graphs (KGs) into a continuous, low-dimensional vector space. It has been shown as an effective tool for integrating knowledge graphs to improve various intelligent applications, such as question answering and information extraction. However, previous KGE models ignore the hidden natural order of knowledge learning on learning the embeddings of entities and relations, leaving room for improvement in their performance. Inspired by the easy-to-hard pattern used in human knowledge learning, this paper proposes a Curriculum learning-based KGE (C-KGE) model, which learns the embeddings of entities and relations from “basic knowledge” to “domain knowledge”. Specifically, a seed set representing the basic knowledge and several knowledge subsets are identified from KG. Then, entity overlap is employed to score the learning difficulty of each subset. Finally, C-KGE trains the entities and relations in each subset according to the learning difficulty score of each subset. C-KGE leverages trained embeddings of the seed set as prior knowledge and learns knowledge subsets iteratively to transfer knowledge between the seed set and subsets, smoothing the learning process of knowledge facts. Experimental results on real-world datasets demonstrate that the proposed model achieves improved embedding performances as well as reducing training time. Our codes and data will be released later.

A novel model for relation prediction in knowledge graphs exploiting semantic and structural feature integration

Relation prediction is a critical task in knowledge graph completion and associated downstream tasks that rely on knowledge representation. Previous studies indicate that both structural features and semantic information are meaningful for predicting missing relations in knowledge graphs. This has led to the development of two types of methods: structure-based methods and semantics-based methods. Since these two approaches represent two distinct learning paradigms, it is difficult to fully utilize both sets of features within a single learning model, especially deep features. As a result, existing studies usually focus on only one type of feature. This leads to an insufficient representation of knowledge in current methods and makes them prone to overlooking certain patterns when predicting missing relations. In this study, we introduce a novel model, RP-ISS, which combines deep semantic and structural features for relation prediction. The RP-ISS model utilizes a two-part architecture, with the first component being a RoBERTa module that is responsible for extracting semantic features from entity nodes. The second part of the system employs an edge-based relational message-passing network designed to capture and interpret structural information within the data. To alleviate the computational burden of the message-passing network on the RoBERTa module during the sampling process, RP-ISS introduces a node embedding memory bank, which updates asynchronously to circumvent excessive computation. The model was assessed on three publicly accessible datasets (WN18RR, WN18, and FB15k-237), and the results revealed that RP-ISS surpasses all baseline methods across all evaluation metrics. Moreover, RP-ISS showcases robust performance in graph inductive learning.

Entity and Relation Linking for Knowledge Graph Question Answering Using Gradual Searching

Knowledge graph question answering (KGQA) systems have an important role in retrieving data from a knowledge graph (KG). With the system, regular users can access data from a KG without the need to construct a formal SPARQL query. KGQA systems receive a natural language question (NLQ) and translate it into a SPARQL query through three main tasks, namely, entity and relation detection, entity and relation linking, and query construction. However, the translation is not trivial due to lexical gaps and entity ambiguity that may occur during entity or relation linking. This research proposed an approach based on multiclass classification of NLQ whose entity occurrences are detected into categories based on KG relations to address the lexical gap challenge. Next, to solve the entity ambiguity challenge, this research proposed a three-stage searching procedure to determine appropriate KG entities associated with the NLQ entities, given the correspondence between the NLQ and a particular KG relation. This three-stage searching consisted of text-based searching, vector-based searching, and entity and relation pairing. The proposed approach was evaluated on the SimpleQuestions and LC-QuAD 2.0 datasets. The experiments demonstrated that the proposed approach outperformed the state-of-the-art baseline. For the relation linking task, the proposed approach reached 89.87% and 74.83% recall for the SimpleQuestions and LC-QuAD 2.0, respectively. This approach also achieved 91.74% and 61.96% recall on the entity linking tasks for the SimpleQuestions and LC-QuAD 2.0, respectively.

Knowledge Graph Embedding: A Survey from the Perspective of Representation Spaces

Knowledge graph embedding (KGE) is an increasingly popular technique that aims to represent entities and relations of knowledge graphs into low-dimensional semantic spaces for a wide spectrum of applications such as link prediction, knowledge reasoning and knowledge completion. In this article, we provide a systematic review of existing KGE techniques based on representation spaces. Particularly, we build a fine-grained classification to categorise the models based on three mathematical perspectives of the representation spaces: (1) algebraic perspective, (2) geometric perspective and (3) analytical perspective. We introduce the rigorous definitions of fundamental mathematical spaces before diving into KGE models and their mathematical properties. We further discuss different KGE methods over the three categories, as well as summarise how spatial advantages work over different embedding needs. By collating the experimental results from downstream tasks, we also explore the advantages of mathematical space in different scenarios and the reasons behind them. We further state some promising research directions from a representation space perspective, with which we hope to inspire researchers to design their KGE models as well as their related applications with more consideration of their mathematical space properties.

Learnable convolutional attention network for knowledge graph completion

Recently, graph convolutional networks (GCNs) and graph attention networks (GATs) have been used extensively in knowledge graph completion (KGC), which aims to solve the incompleteness of knowledge graphs (KGs). However, both GCNs and GATs have limitations in the KGC task, and the best method is analyzing the neighbors of each entity (pre-validating), while this process is prohibitively expensive. Furthermore, relations in KGs have specific semantics and should be considered when aggregating neighbor information (message passing). To address the above limitations, we propose a learnable convolutional attention network for knowledge graph completion named LCA-KGC. LCA-KGC introduces a knowledge graph convolutional attention network using a convolution operation before the attention mechanism to ensure structural information acquisition and avoid redundant information stacking. Furthermore, to complete the autonomous switching of GNNs types and eliminate the necessity of pre-validating the local structure of KGs, LCA-KGC designs a learnable knowledge graph convolutional attention network by comprising three types of GNNs in one learnable formulation. Moreover, a learnable message function is proposed to emphasize relational semantics when aggregating neighbor information. Extensive experiments on standard KG datasets validate the effectiveness of the proposed innovations, and LCA-KGC achieves state-of-the-art (SOTA) performance compared to existing approaches (e.g., compared to SOTA approaches, LCA-KGC improves MRR from 0.360 to 0.372 on FB15k-237 dataset, and Hits@3 from 0.561 to 0.581 on YAGO3-10 dataset).

SDFormer: A shallow-to-deep feature interaction for knowledge graph embedding

Inferring missing information from current facts in a knowledge graph (KG) is the target of the link prediction task. Currently, existing methods embed the entities and relations of KG as a whole into a low-dimensional vector space. Nonetheless, they ignore the multi-level interactions (shallow interactions, deep interactions) among the finer-grained sub-features of entities and relations. To overcome these limitations, we present a shallow-to-deep feature interaction for knowledge graph embedding (SDFormer). It takes into account the interpretability of sub-feature tokens of entities and relations and learns shallow-to-deep interaction information between entities and relations at a more fine-grained level. Specifically, entity and relation vectors are decomposed into sub-features to represent multi-dimensional information. Then, a shallow-to-deep feature interaction method is designed to capture multi-level interactions between entities and relations. This process enriches the feature representation by modeling the interaction between sub-features. Finally, a 1-X scoring function is utilized to calculate the score of each knowledge triplet. The experimental results on several benchmark datasets show that SDFormer obtains competitive performance results and more efficient training efficiency on other comparative models and because of the shallow-to-deep feature interaction between entities and relations.

Complete feature learning and consistent relation modeling for few-shot knowledge graph completion

Few-shot knowledge graph completion focuses on predicting unseen facts of long-tail relations in knowledge graphs with only few reference sets. The key challenge for tackling this task is how to represent the complete entity features under low data regime conditions and further build the relation scoring function of the triplet for prediction. However, existing works mainly focus on aggregating entity representations and seriously ignore the process of consistent relation modeling, resulting in unsatisfactory performance on sparse neighbors and complex relations modeling. To address the issues, this paper designs a two-branch feature extractor to capture complementary and complete representation of entities for differentiating the few examples, where each branch focuses on diverse aspect of the entity features. Furthermore, we apply a diversity loss based on the minimization of cosine similarity is applied between the two-branch feature extractors to encourage the two-branch to learn complementary features. Conditioned on the entity features, we further incorporate the structural relation representation into the semantic relation learning to keep the consistent with triplet scoring function, and consider the consistency issue of various structural relation modeling between training and test generalization. Empirical results on two public benchmark datasets NELL-One and Wiki-One demonstrate that our approach outperforms the state-of-the-art results, with relative improvements on Hits@10 for 1-shot of 4.8% and 4.4%, respectively, and achieves new state-of-the-art results. Additionally, Extensive experiments also show proficiency in dealing with complex relations and sparse neighbors.

Constructing negative samples via entity prediction for multi-task knowledge representation learning

Knowledge representation learning (KRL) aims at embedding the entities and relations in knowledge graphs (KGs) into vectors by learning. In recent years, several multi-task learning (MTL) frameworks were proposed to KRL and achieved promising results. Nevertheless, most of these models only share the initial embeddings of entities and relations among sub-tasks. We construct hard negative samples for relation prediction (RP) task and triple classification (TC) task by using the similarity matrix from the entity prediction (EP) task. We also share the features in hidden layers of EP task to TC. The proposed framework is applicable to CNN based models for KRL. Experimental results on WN18RR and FB15k-237 datasets show that the proposed framework promotes the performance of EP, RP and TC tasks evidently comparing with the single task models and obtains better results in RP and TC than the state-of-the-art MTL-based models/frameworks. Code of NS-KRL is available online.11https://github.com/fofilix/NS_KRL.

Few-Shot Relation Prediction of Knowledge Graph via Convolutional Neural Network with Self-Attention

Knowledge graph (KG) has become the vital resource for various applications like question answering and recommendation system. However, several relations in KG only have few observed triples, which makes it necessary to develop the method for few-shot relation prediction. In this paper, we propose the Convolutional Neural Network with Self-Attention Relation Prediction (CARP) model to predict new facts with few observed triples. First, to learn the relation property features, we build a feature encoder by using the convolutional neural network with self-attention from the few observed triples rather than background knowledge. Then, by incorporating the learned features, we give an embedding network to learn the representation of incomplete triples. Finally, we give the loss function and training algorithm of our CARP model. Experimental results on three real-world datasets show that our proposed method improves Hits@10 by 48% on average over the state-of-the-art competitors.

Reducing noise-triplets via differentiable sampling for knowledge-enhanced recommendation with collaborative signal guidance

Knowledge Graph (KG) is widely used for recommendation tasks due to its rich semantic information and external structure. Current knowledge graph recommendation models are insufficient for the learning of user–item interactive behaviors as well as external knowledge, and ignore the noise information in KG. To alleviate these limitations, we present a Collaborative Relation-aware Attention Network with Differentiable sampling (DCRAN) that can fully learn user–item interactions and knowledge graphs, and accurately select recommendation-related entities in KG. Specifically, DCRAN explicitly learns the embeddings of users and items and encodes them into collaborative interaction signals, which are used as guidance signals to extract knowledge. Furthermore, DCRAN emphasizes the importance of KG relations and constructs a relation-aware attention network to learn the representation of items in KG. Most importantly, DCRAN applies a differentiable sampling strategy on entities, which can reduce triplets that have a negative effect on recommendation. Experimental results on three public datasets manifest the effectiveness of DCRAN.

Meta-Learning Based Dynamic Adaptive Relation Learning for Few-Shot Knowledge Graph Completion

As artificial intelligence gradually steps into cognitive intelligence stage, knowledge graphs (KGs) play an increasingly important role in many natural language processing tasks. Due to the prevalence of long-tail relations in KGs, few-shot knowledge graph completion (KGC) for link prediction of long-tail relations has gradually become a hot research topic. Current few-shot KGC methods mainly focus on the static representation of surrounding entities to explore the potential semantic features of entities, while ignoring the dynamic properties among entities and the special influence of the long-tail relation on link prediction. In this paper, a new meta-learning based dynamic adaptive relation learning model (DARL) is proposed for few-shot KGC. For obtaining better semantic information of the meta knowledge, the proposed DARL model applies a dynamic neighbor encoder to incorporate neighbor relations into entity embedding. In addition, DARL builds attention mechanism based fusion strategy for different attributes of the same relation to further enhance the relation-meta learning ability. We evaluate our DARL model on two public benchmark datasets NELL-One and WIKI-One for link prediction. Extensive experimental results indicate that our DARL outperforms the state-of-the-art models with an average relative improvement about 23.37%, 32.46% in MRR and [email protected] on NELL-One, respectively.

RelaGraph: Improving embedding on small-scale sparse knowledge graphs by neighborhood relations

Learning a continuous dense low-dimensional representation of knowledge graphs (KGs), known as knowledge graph embedding (KGE), has been viewed as the key to intelligent reasoning for deep learning (DL) and gained much attention in recent years. To address the problem that the current KGE models are generally ineffective on small-scale sparse datasets, we propose a novel method RelaGraph to improve the representation of entities and relations in KGs by introducing neighborhood relations. RelaGraph extends the neighborhood information during entity encoding, and adds the neighborhood relations to mine deeper level of graph structure information, so as to make up for the shortage of information in the generated subgraph. This method can well represent KG components in a vector space in a way that captures the structure of the graph, avoiding underlearning or overfitting. KGE based on RelaGraph is evaluated on a small-scale sparse graph KMHEO, and the MRR reached 0.49, which is 34 percentage points higher than that of the SOTA methods, as well as it does on several other datasets. Additionally, the vectors learned by RelaGraph is used to introduce DL into several KG-related downstream tasks, which achieved excellent results, verifying the superiority of KGE-based methods.

Multi-Aspect Explainable Inductive Relation Prediction by Sentence Transformer

Recent studies on knowledge graphs (KGs) show that path-based methods empowered by pre-trained language models perform well in the provision of inductive and explainable relation predictions. In this paper, we introduce the concepts of relation path coverage and relation path confidence to filter out unreliable paths prior to model training to elevate the model performance. Moreover, we propose Knowledge Reasoning Sentence Transformer (KRST) to predict inductive relations in KGs. KRST is designed to encode the extracted reliable paths in KGs, allowing us to properly cluster paths and provide multi-aspect explanations. We conduct extensive experiments on three real-world datasets. The experimental results show that compared to SOTA models, KRST achieves the best performance in most transductive and inductive test cases (4 of 6), and in 11 of 12 few-shot test cases.

TuckerDNCaching: high-quality negative sampling with tucker decomposition

Knowledge Graph Embedding (KGE) translates entities and relations of knowledge graphs (KGs) into a low-dimensional vector space, enabling an efficient way of predicting missing facts. Generally, KGE models are trained with positive and negative examples, discriminating positives against negatives. Nevertheless, KGs contain only positive facts; KGE training requires generating negatives from non-observed ones in KGs, referred to as negative sampling. Since KGE models are sensitive to inputs, negative sampling becomes crucial, and the quality of the negatives becomes critical in KGE training. Generative adversarial networks (GAN) and self-adversarial methods have recently been utilized in negative sampling to address the vanishing gradients observed with early negative sampling methods. However, they introduce the problem of false negatives with high probability. In this paper, we extend the idea of reducing false negatives by adopting a Tucker decomposition representation, i.e., TuckerDNCaching, to enhance the semantic soundness of latent relations among entities by introducing a relation feature space. TuckerDNCaching ensures the quality of generated negative samples, and the experimental results reflect that our proposed negative sampling method outperforms the existing state-of-the-art negative sampling methods.

Auction-Based Learning for Question Answering over Knowledge Graphs

Knowledge graphs are graph-based data models which can represent real-time data that is constantly growing with the addition of new information. The question-answering systems over knowledge graphs (KGQA) retrieve answers to a natural language question from the knowledge graph. Most existing KGQA systems use static knowledge bases for offline training. After deployment, they fail to learn from unseen new entities added to the graph. There is a need for dynamic algorithms which can adapt to the evolving graphs and give interpretable results. In this research work, we propose using new auction algorithms for question answering over knowledge graphs. These algorithms can adapt to changing environments in real-time, making them suitable for offline and online training. An auction algorithm computes paths connecting an origin node to one or more destination nodes in a directed graph and uses node prices to guide the search for the path. The prices are initially assigned arbitrarily and updated dynamically based on defined rules. The algorithm navigates the graph from the high-price to the low-price nodes. When new nodes and edges are dynamically added or removed in an evolving knowledge graph, the algorithm can adapt by reusing the prices of existing nodes and assigning arbitrary prices to the new nodes. For subsequent related searches, the “learned” prices provide the means to “transfer knowledge” and act as a “guide”: to steer it toward the lower-priced nodes. Our approach reduces the search computational effort by 60% in our experiments, thus making the algorithm computationally efficient. The resulting path given by the algorithm can be mapped to the attributes of entities and relations in knowledge graphs to provide an explainable answer to the query. We discuss some applications for which our method can be used.

A novel KG-based recommendation model via relation-aware attentional GCN

Leveraging knowledge graphs (KGs) to enhance recommender systems has gained considerable attention, with researchers obtaining user preferences by aggregating entity pairs with explicit relations in KGs via graph convolutional networks (GCNs). Existing approaches currently overlook many entity pairs without relations, which, however, may have potentially useful information. To address this issue, we propose a novel relation-aware attentional GCN (RAAGCN) with the following improvements over vanilla GCNs: (1) it aggregates all entity pairs with and without explicit relations and (2) it distinguishes the importance of different relational context information. Based on the proposed RAAGCN, we further propose a user preference and item attractiveness capturing model (UPIACM) for KG-based recommendation. In the UPIACM, the user preference is decomposed into interest and rating preferences. The interest preference is the user’s interest taste toward the items with specific features, while the rating preference reflects the intention of rating high or low. Additionally, our model accounts for item attractiveness, which reflects an item’s popularity among users. Additionally, we incorporate a gated filtering mechanism to further improve our model’s performance. Through extensive experiments, we show that the proposed UPIACM outperforms state-of-the-art baseline methods.

A Novel Embedding Model for Knowledge Graph Entity Alignment Based on Graph Neural Networks

The objective of the entity alignment (EA) task is to identify entities with identical semantics across distinct knowledge graphs (KGs) situated in the real world, which has garnered extensive recognition in both academic and industrial circles. Within this paper, a pioneering entity alignment framework named PCE-HGTRA is proposed. This framework integrates the relation and property information from varying KGs, along with the heterogeneity information present within the KGs. Firstly, by learning embeddings, this framework captures the similarity that exists between entities present across diverse KGs. Additionally, property triplets in KGs are used to generate property character-level embeddings, facilitating the transfer of entity embeddings from two distinct KGs onto an identical space. Secondly, the framework strengthens the property character-level embeddings using the transitivity rule to increase the count of entity properties. Then, in order to effectively capture the heterogeneous features in the entity neighborhoods, a heterogeneous graph transformer with relation awareness is designed to model the heterogeneous relations in KGs in the framework. Finally, comparative experimental results on four widely recognized real-world datasets demonstrate that PCE-HGTRA performs exceptionally well. In fact, its Hits@1 performance exceeds the best baseline by 7.94%, outperforming seven other state-of-the-art methods.

Bilinear Scoring Function Search for Knowledge Graph Learning.

Learning embeddings for entities and relations in knowledge graph (KG) have benefited many downstream tasks. In recent years, scoring functions, the crux of KG learning, have been human designed to measure the plausibility of triples and capture different kinds of relations in KGs. However, as relations exhibit intricate patterns that are hard to infer before training, none of them consistently perform the best on benchmark tasks. In this paper, inspired by the recent success of automated machine learning (AutoML), we search bilinear scoring functions for different KG tasks through the AutoML techniques. However, it is non-trivial to explore domain-specific information here. We first set up a search space for AutoBLM by analyzing existing scoring functions. Then, we propose a progressive algorithm (AutoBLM) and an evolutionary algorithm (AutoBLM+), which are further accelerated by filter and predictor to deal with the domain-specific properties for KG learning. Finally, we perform extensive experiments on benchmarks in KG completion, multi-hop query, and entity classification tasks. Empirical results show that the searched scoring functions are KG dependent, new to the literature, and outperform the existing scoring functions. AutoBLM+ is better than AutoBLM as the evolutionary algorithm can flexibly explore better structures in the same budget.

An attention‐based representation learning model for multiple relational knowledge graph

AbstractKnowledge graph embedding models are used to learn low‐dimensional representations of entities and relations in knowledge graphs. In this paper, we propose Multi‐RAttE, an attention‐based learning method for multiple relational knowledge graph embedding representation, which divides the information transfer in the knowledge graph into cross‐relational information transfer and relation‐specific information transfer, and divides the embedding of knowledge graph entities into structural embedding and multi‐relational embedding for joint learning. To objectively analyse the performance of the Multi‐RAttE model, we select two typical datasets and different representative baseline models for experimental evaluation on several tasks such as link prediction, multi‐relation prediction and node classification. The experimental results show that the Multi‐RAttE model improves 8% over the state‐of‐the‐art model Composition‐based Multi‐Relational Graph Convolutional Networks (CompGCN) in terms of Hits@1 metric on the link prediction task on FB15k‐237 dataset; on the multi‐relation prediction task, the accuracy improves by 1.8% and 3.7% in the auc metric and F1 metric, respectively. The experimental results have proved that the Multi‐RAttE model can effectively perform the representation of multiple relations.