Real-world Knowledge Graphs Research Articles

Dual De-confounded Causal Intervention method for knowledge graph error detection

Due to the Knowledge Graph (KG) construction process, erroneous triples are virtually inevitable to be introduced into real-world KGs. Since these errors hinder the expressiveness and applicability of KGs, the development of knowledge graph error detection (KGED) methods is necessary. Despite the overall effectiveness of current KGED methods, their capacity to identify challenging errors is limited. In this work, we conduct empirical studies and find that previous works introduce structural and semantic bias, impeding the identification of erroneous triples, especially in challenging cases. To address this issue, we design a causal graph for the KGED task and propose a Dual De-confounded Causal Intervention (DuDCI) method for debiasing. Firstly, DuDCI utilizes the neighborhood and textual descriptions of triples to calculate their graph and text embeddings. Next, a Causal De-confounded Module is constructed to mitigate the impact of shortcuts caused by the bias through the front-door adjustment. Furthermore, we introduce Disentanglement Constraints to disentangle the information expressed by each embedding, thereby facilitating further bias mitigation. Experimental results on three widely used KGED datasets validate the effectiveness of DuDCI and demonstrate that DuDCI outperforms current KGED methods, with an improvement of at least 2.2%, especially in more challenging noise scenarios.

BGAT-CCRF: A novel end-to-end model for knowledge graph noise correction

Knowledge graph (KG) noise correction aims to select suitable candidates to correct the noises in KGs. Most of the existing studies have limited performance in repairing the noisy triple that contains more than one incorrect entity or relation, which significantly constrains their implementation in real-world KGs. To overcome this challenge, we propose a novel end-to-end model (BGAT-CCRF) that achieves better noise correction results. Specifically, we construct a balanced-based graph attention model (BGAT) to learn the features of nodes in triples’ neighborhoods and capture the correlation between nodes based on their position and frequency. Additionally, we design a constrained conditional random field model (CCRF) to select suitable candidates guided by three constraints for correcting one or more noises in the triple. In this way, BGAT-CCRF can select multiple candidates from a smaller domain to repair multiple noises in triples simultaneously, rather than selecting candidates from the whole KG to repair noisy triples as traditional methods do, which can only repair one noise in the triple at a time. The effectiveness of BGAT-CCRF is validated by the KG noise correction experiment. Compared with the state-of-the-art models, BGAT-CCRF improves the fMRR metric by 3.58% on the FB15K dataset. Hence, it has the potential to facilitate the implementation of KGs in the real world.

Manufacturing resilience through disruption mitigation using attention-based consistently-attributed graph embedded decision support system

With global supply chains experiencing significant disruptions, there is increasing emphasis on enhancing manufacturing resilience by mitigating supply chain-propagated impacts on production entities in a holistic manner. However, existing mitigative strategies typically rely on implicit experience, which may be biased and suboptimal. Furthermore, the lack of consideration towards the chain of custody, multi-disruption management, and transportation networks results in unintuitive systems. To overcome these challenges, this study proposes a graph embedding-based mitigation decision support system (GEM-DSS) using a heterogeneous industrial knowledge graph to facilitate disruption management. Addressing the prevailing industrial challenge of knowledge representations and graph incompleteness, an attention-based consistently-attributed graph embedding (ACAGE) model is proposed, with the capability to learn and factor in multi-attributes and multi-structures within the knowledge graph. Through supplier selection and material substitution mitigation strategies, two computational pipelines are designed as part of a dual-stage search mechanism to generate feasible solutions. The experimental results of the ACAGE model for knowledge graph completion outperform several attributed graph embedding baselines, achieving 0.999 AUC-ROC and 0.990 AUC-PR on the constructed real-world industrial knowledge graph. The functionality and applicability of the GEM-DSS are demonstrated in an automotive case study with dynamic considerations via a cognitive computing methodology. This study aims to provide valuable insights for both academia and industry towards enabling manufacturing continuity in the face of global supply chain challenges.

GLaM: Fine-Tuning Large Language Models for Domain Knowledge Graph Alignment via Neighborhood Partitioning and Generative Subgraph Encoding

Integrating large language models with knowledge graphs derived from domain-specific data represents an important advancement towards more powerful and factual reasoning. As these models grow more capable, it is crucial to enable them to perform multi-step inferences over real-world knowledge graphs while minimizing hallucination. While large language models excel at conversation and text generation, their ability to reason over domain-specialized graphs of interconnected entities remains limited. For example, can we query a model to identify the optimal contact in a professional network for a specific goal, based on relationships and attributes in a private database? The answer is no – such capabilities lie beyond current methods. However, this question underscores a critical technical gap that must be addressed. Many high-value applications in areas such as science, security, and e-commerce rely on proprietary knowledge graphs encoding unique structures, relationships, and logical constraints. We introduce a fine-tuning framework for developing Graph-aligned Language Models (GaLM) that transforms a knowledge graph into an alternate text representation with labeled question-answer pairs. We demonstrate that grounding the models in specific graph-based knowledge expands the models’ capacity for structure-based reasoning. Our methodology leverages the large-language model's generative capabilities to create the dataset and proposes an efficient alternate to retrieval-augmented generation styled methods.

Multi-Filter soft shrinkage network for knowledge graph embedding

Incompleteness is a prominent issue pervasive in real-world knowledge graphs, and link prediction techniques, which utilize known facts to forecast missing or unknown links, have emerged as a critical technology for knowledge graph completion. Recently, convolutional neural network-based knowledge graph embedding (KGE) models have achieved breakthroughs in the field of link prediction by increasing the number of interactions between entities and relations. However, current models mainly emphasize stacking complex architectures to capture more interaction features, ignoring the fact that, as the number of features increases significantly, irrelevant features might overshadow important ones, thus impacting the performance of link prediction. To tackle this problem, this paper introduces a novel KGE model that is based on a multi-filter soft shrinkage network (MFSSN). First, this model adaptively constructs filters based on entity embedding and relationship embedding, thereby increasing the interactions between entities and relationships. Secondly, the soft shrinkage function is innovatively introduced into the model, and a specialized soft shrinkage sub-network is innovatively designed to effectively eliminate noise features and improve attention to important features. Finally, the attention mechanism is added to the network to enhance important features and suppress useless features, improving the ability to utilize features, and thereby enhancing model performance. Through extensive experiments on five benchmark datasets of different sizes, the strong performance and generalization ability of the proposed model are proven. Compared with contrasting methods, it achieves performance leadership in almost all indicators.

Anchoring Path for Inductive Relation Prediction in Knowledge Graphs

Aiming to accurately predict missing edges representing relations between entities, which are pervasive in real-world Knowledge Graphs (KGs), relation prediction plays a critical role in enhancing the comprehensiveness and utility of KGs. Recent research focuses on path-based methods due to their inductive and explainable properties. However, these methods face a great challenge when lots of reasoning paths do not form Closed Paths (CPs) in the KG. To address this challenge, we propose Anchoring Path Sentence Transformer (APST) by introducing Anchoring Paths (APs) to alleviate the reliance of CPs. Specifically, we develop a search-based description retrieval method to enrich entity descriptions and an assessment mechanism to evaluate the rationality of APs. APST takes both APs and CPs as the inputs of a unified Sentence Transformer architecture, enabling comprehensive predictions and high-quality explanations. We evaluate APST on three public datasets and achieve state-of-the-art (SOTA) performance in 30 of 36 transductive, inductive, and few-shot experimental settings.

AMT-CDR: A Deep Adversarial Multi-channel Transfer Network for Cross-domain Recommendation

Recommender systems are one of the most successful applications of using AI for providing personalized e-services to customers. However, data sparsity is presenting enormous challenges that are hindering the further development of advanced recommender systems. Although cross-domain recommendation partly overcomes data sparsity by transferring knowledge from a source domain with relatively dense data to augment data in the target domain, the current methods do not handle heterogeneous data very well. For example, using today’s cross-domain transfer learning schemes with data comprising clicks, ratings, user reviews, item meta data, and knowledge graphs will likely result in a poorly-performing model. User preferences will not be comprehensively profiled, and accurate recommendations will not be generated. To solve these three challenges – i.e., handling heterogeneous data, avoiding negative transfer, and dealing with data sparsity – we designed a new end-to-end deep a dversarial m ulti-channel t ransfer network for c ross- d omain r ecommendation named AMT-CDR. Heterogeneous data is handled by constructing a cross-domain graph based on real-world knowledge graphs – we used Freebase and YAGO. Negative transfer is prevented through an adversarial learning strategy that maintains consistency across the different data channels. And data sparsity is addressed with an end-to-end neural network that considers data across multiple channels and generates accurate recommendations by leveraging knowledge from both the source and target domains. Extensive experiments on three dual-target cross-domain recommendation tasks demonstrate the superiority of AMT-CDR compared to eight state-of-the-art methods. All source code is available at https://github.com/bjtu-lucas-nlp/AMT-CDR.

Adaptive Prototype Interaction Network for Few-Shot Knowledge Graph Completion.

Few-shot knowledge graph completion (FKGC), which aims to infer new triples for a relation using only a few reference triples of the relation, has attracted much attention in recent years. Most existing FKGC methods learn a transferable embedding space, where entity pairs belonging to the same relations are close to each other. In real-world knowledge graphs (KGs), however, some relations may involve multiple semantics, and their entity pairs are not always close due to having different meanings. Hence, the existing FKGC methods may yield suboptimal performance when handling multiple semantic relations in the few-shot scenario. To solve this problem, we propose a new method named adaptive prototype interaction network (APINet) for FKGC. Our model consists of two major components: 1) an interaction attention encoder (InterAE) to capture the underlying relational semantics of entity pairs by modeling the interactive information between head and tail entities and 2) an adaptive prototype net (APNet) to generate relation prototypes adaptive to different query triples by extracting query-relevant reference pairs and reducing the data inconsistency between support and query sets. Experimental results on two public datasets demonstrate that APINet outperforms several state-of-the-art FKGC methods. The ablation study demonstrates the rationality and effectiveness of each component of APINet.

Multi-view semantic enhancement model for few-shot knowledge graph completion

In recent years, few-shot knowledge graph completion (FKGC) has gained popularity as a solution to the long-tail distribution problem of real-world knowledge graphs (KGs). The previous knowledge graph completion (KGC) models obtain triple representation merely relying on the structure view, ignoring the valuable semantic knowledge. In this paper, we propose a multi-view framework for few-shot relation learning to address the issue. Specifically, based the structural information of the graph obtained using the structure view, we add the text view and the commonsense view. The text view employs text descriptions of entities and relations to obtain a richer semantic representation. The commonsense view performs high-quality negative sampling based on complex relations. Moreover, commonsense semantic constraints are invoked to suppress the overfitting caused by the complexity of the relation matrix. Extensive experiments show that our model outperforms state-of-the-art FKGC methods on the frequently-used benchmark datasets FB15k237-One and NELL-One.

HPRE: Leveraging hierarchy-aware paired relation vectors for knowledge graph embedding

Knowledge graphs exhibit a typical hierarchical structure and find extensive applications in various artificial intelligence domains. However, large-scale knowledge graphs need to be completed, which limits the performance of knowledge graphs in downstream tasks. Knowledge graph embedding methods have emerged as a primary solution to enhance knowledge graph completeness. These methods aim to represent entities and relations as low-dimensional vectors, focusing on handling relation patterns and multi-relation types. Researchers need to pay more attention to the crucial feature of hierarchical relationships in real-world knowledge graphs. We propose a novel knowledge graph embedding model called Hierarchy-Aware Paired Relation Vectors Knowledge Graph Embedding (HPRE) to bridge this gap. By leveraging the power of 2D coordinates, HPRE adeptly model relation patterns, multi-relation types, and hierarchical features in the knowledge graph. Specifically, HPRE employs paired relation vectors to capture the distinct characteristics of head and tail entities, facilitating a better fit for relational patterns and multi-relation scenarios. Additionally, HPRE employs angular coordinates to differentiate entities at various levels of the hierarchy, effectively representing the hierarchical nature of the knowledge graph. The experimental results show that the HPRE model can effectively learn the hierarchical features of the knowledge graph and achieve state-of-the-art experimental results on multiple real-world datasets for the link prediction task.

Transfer Learning across Graph Convolutional Networks: Methods, Theory, and Applications

Graph neural networks have been widely used for learning representations of nodes for many downstream tasks on graph data. Existing models were designed for the nodes on a single graph, which would not be able to utilize information across multiple graphs. The real world does have multiple graphs where the nodes are often partially aligned . For examples, knowledge graphs share a number of named entities though they may have different relation schema; collaboration networks on publications and awarded projects share some researcher nodes who are authors and investigators, respectively; people use multiple web services, shopping, tweeting, rating movies, and some may register the same email account across the platforms. In this paper, we propose partially aligned graph convolutional networks to learn node representations across the models. We provide multiple methods such as model sharing, regularization, and alignment reconstruction, as well as theoretical analysis to positively transfer knowledge across the set of partially aligned nodes. Extensive experiments on real-world knowledge graphs, collaboration networks, and bipartite rating graphs show the superior performance of our proposed methods on relation classification, link prediction, and item recommendation.

EASC: An exception-aware semantic compression framework for real-world knowledge graphs

Knowledge graphs (KGs) have achieved great success in many real applications, and great efforts have been dedicated to constructing larger knowledge graphs. An obvious trend in KG construction is that the KGs become ever-increasingly bigger. However, we argue that constructing a KG by directly inserting more triples may harm the performance of the KG, and one possible solution is KG compression. In this paper, we propose an exception-aware semantic lossless compression framework EASC to compress a KG. Since many triples can be inferred from other triples with semantic rules, we remove the triples that can be inferred and store the rules and exception cases. Specifically, we formalize the lossless compression problem as a weighted set cover problem, which is NP-hard, and propose a semantic lossless compression algorithm to get an approximation result. We conduct extensive experiments on seven real-world large-scale KGs. The results show that EASC achieves state-of-the-art performance in semantic compression methods. Furthermore, by combining EASC as an independent module with syntactic compression methods, we achieve state-of-the-art performance in lossless compression methods.

A type-augmented knowledge graph embedding framework for knowledge graph completion

Knowledge graphs (KGs) are of great importance to many artificial intelligence applications, but they usually suffer from the incomplete problem. Knowledge graph embedding (KGE), which aims to represent entities and relations in low-dimensional continuous vector spaces, has been proved to be a promising approach for KG completion. Traditional KGE methods only concentrate on structured triples, while paying less attention to the type information of entities. In fact, incorporating entity types into embedding learning could further improve the performance of KG completion. To this end, we propose a universal Type-augmented Knowledge graph Embedding framework (TaKE) which could utilize type features to enhance any traditional KGE models. TaKE automatically captures type features under no explicit type information supervision. And by learning different type representations of each entity, TaKE could distinguish the diversity of types specific to distinct relations. We also design a new type-constrained negative sampling strategy to construct more effective negative samples for the training process. Extensive experiments on four datasets from three real-world KGs (Freebase, WordNet and YAGO) demonstrate the merits of our proposed framework. In particular, combining TaKE with the recent tensor factorization KGE model SimplE can achieve state-of-the-art performance on the KG completion task.

KE-X: Towards subgraph explanations of knowledge graph embedding based on knowledge information gain

Over the past years, knowledge graph embedding approaches have proven effective for knowledge graph completion tasks. However, most existing models are either built on a certain embedding space or black-box neural networks, making it hard to access explanations for prediction results and resulting in limited explainability. In this paper, we propose to leverage information entropy to quantify the importance of explanation candidates, then build a framework KE-X, for explaining results from knowledge graph embedding approaches by generating explainable subgraphs. Specifically, by performing a modified message passing mechanism on a partially masked knowledge subgraph and maximizing knowledge information gain, KE-X can extract the most valuable subgraph explanation for a link prediction query. To evaluate KE-X, we conduct experiments on three real-world knowledge graphs with two representative KGE models, TransE and DistMult. Both quantitative and case study results show that our framework can extract high-quality explanations.

Incorporating logic rules with textual representations for interpretable knowledge graph reasoning

Reasoning on knowledge graphs (KGs) is significant for downstream applications, such as question answering and information extraction. On the basis of using factual triples in KGs, learning logic rules has been explored to handle KG reasoning in an interpretable pattern. Some rule learning methods simply apply thresholds to reduce the exponential complexity caused by the vast candidate rules derived from KGs. Others are designed to prune candidates on the basis of embedding facts in the KG. However, these methods only consider the factual triples, which mainly contain structural information, and ignore sufficient semantic information in real-world KGs. This situation weakens the methods in searching for high-quality rules. To this end, we propose a joint method by incorporating logic rules with textual representations for interpretable KG reasoning, named LoTus. Firstly, LoTus obtains the structural embeddings with only factual triples, and the textual embeddings with descriptions of entities, which are later integrated into the representations of entities and relations. Secondly, top candidate rules are selected by a proposed joint pruning strategy incorporating structural and textual representations in KGs. Finally, LoTus captures high-quality rules and removes low-quality ones by a filtering process with metrics of rules. It reduces the time complexity on KG reasoning by learning rules and provides interpretability simultaneously. Extensive experiments on different datasets show that LoTus obtains competitive performance comparing to baselines on KG reasoning metrics Hits@1, Hits@10 and mean reciprocal rank (MRR), and achieves significant superiority among the recent rule learning methods.

Knowledge graph embedding methods for entity alignment: experimental review

In recent years, we have witnessed the proliferation of knowledge graphs (KG) in various domains, aiming to support applications like question answering, recommendations, etc. A frequent task when integrating knowledge from different KGs is to find which subgraphs refer to the same real-world entity, a task largely known as the Entity Alignment. Recently, embedding methods have been used for entity alignment tasks, that learn a vector-space representation of entities which preserves their similarity in the original KGs. A wide variety of supervised, unsupervised, and semi-supervised methods have been proposed that exploit both factual (attribute based) and structural information (relation based) of entities in the KGs. Still, a quantitative assessment of their strengths and weaknesses in real-world KGs according to different performance metrics and KG characteristics is missing from the literature. In this work, we conduct the first meta-level analysis of popular embedding methods for entity alignment, based on a statistically sound methodology. Our analysis reveals statistically significant correlations of different embedding methods with various meta-features extracted by KGs and rank them in a statistically significant way according to their effectiveness across all real-world KGs of our testbed. Finally, we study interesting trade-offs in terms of methods’ effectiveness and efficiency.

Lifelong Embedding Learning and Transfer for Growing Knowledge Graphs

Existing knowledge graph (KG) embedding models have primarily focused on static KGs. However, real-world KGs do not remain static, but rather evolve and grow in tandem with the development of KG applications. Consequently, new facts and previously unseen entities and relations continually emerge, necessitating an embedding model that can quickly learn and transfer new knowledge through growth. Motivated by this, we delve into an expanding field of KG embedding in this paper, i.e., lifelong KG embedding. We consider knowledge transfer and retention of the learning on growing snapshots of a KG without having to learn embeddings from scratch. The proposed model includes a masked KG autoencoder for embedding learning and update, with an embedding transfer strategy to inject the learned knowledge into the new entity and relation embeddings, and an embedding regularization method to avoid catastrophic forgetting. To investigate the impacts of different aspects of KG growth, we construct four datasets to evaluate the performance of lifelong KG embedding. Experimental results show that the proposed model outperforms the state-of-the-art inductive and lifelong embedding baselines.

Learning Representations of Bi-level Knowledge Graphs for Reasoning beyond Link Prediction

Knowledge graphs represent known facts using triplets. While existing knowledge graph embedding methods only consider the connections between entities, we propose considering the relationships between triplets. For example, let us consider two triplets T1 and T2 where T1 is (Academy_Awards, Nominates, Avatar) and T2 is (Avatar, Wins, Academy_Awards). Given these two base-level triplets, we see that T1 is a prerequisite for T2. In this paper, we define a higher-level triplet to represent a relationship between triplets, e.g., where PrerequisiteFor is a higher-level relation. We define a bi-level knowledge graph that consists of the base-level and the higher-level triplets. We also propose a data augmentation strategy based on the random walks on the bi-level knowledge graph to augment plausible triplets. Our model called BiVE learns embeddings by taking into account the structures of the base-level and the higher-level triplets, with additional consideration of the augmented triplets. We propose two new tasks: triplet prediction and conditional link prediction. Given a triplet T1 and a higher-level relation, the triplet prediction predicts a triplet that is likely to be connected to T1 by the higher-level relation, e.g., . The conditional link prediction predicts a missing entity in a triplet conditioned on another triplet, e.g., . Experimental results show that BiVE significantly outperforms all other methods in the two new tasks and the typical base-level link prediction in real-world bi-level knowledge graphs.

Anchors-Based Incremental Embedding for Growing Knowledge Graphs

Knowledge graph embedding aims to transform the entities and relations of triplets into the low-dimensional vectors. Previous methods are oriented towards the static knowledge graphs, in which all entities and relations are assumed to be known and only some unknown triplets need to be predicted. However, the real-world knowledge graphs can grow dynamically, and some new knowledge are often added. To embed the new knowledge into the space of original knowledge graph, the classic models have to perform the entire re-embedding with including the new and original knowledge. This causes heavy computational burden for embedding. To address this problem, this study proposes a new model of anchors-based incremental embedding (ABIE) to implement the dynamical embedding for the growing knowledge graph. According to ABIE, every knowledge graph has some key entities, called anchors, which can fix the embedding space of knowledge graph. When some new knowledge is added into the graph, only a few updated entities and relations are embedded into the embedding space with the help of anchors, and the entire re-embedding on the whole graph is not necessary. By this way, the computational burden of embedding caused by the growth of knowledge graph is reduced significantly.

Locality-aware subgraphs for inductive link prediction in knowledge graphs

Recent methods for inductive reasoning on Knowledge Graphs (KGs) transform the link prediction problem into a graph classification task. They first extract a subgraph around each target link based on the k-hop neighborhood of the target entities, encode the subgraphs using a Graph Neural Network (GNN), then learn a function that maps subgraph structural patterns to link existence. Although these methods have witnessed great successes, increasing k often leads to an exponential expansion of the neighborhood, thereby degrading the GNN expressivity due to oversmoothing. In this paper, we formulate the subgraph extraction as a local clustering procedure that aims at sampling tightly-related subgraphs around the target links, based on a personalized PageRank (PPR) approach. Empirically, on three real-world KGs, we show that reasoning over subgraphs extracted by PPR-based local clustering can lead to a more accurate link prediction model than relying on neighbors within fixed hop distances. Furthermore, we investigate graph properties such as average clustering coefficient and node degree, and show that there is a relation between these and the performance of subgraph-based link prediction.