Knowledge Base Completion Research Articles

Complete inference via knowledge Petri nets and resolution rules

A novel inference approach is presented based on knowledge Petri nets and resolution rules. First, a knowledge base is modeled as a knowledge Petri net, where logical clauses are represented by monitor places, called semantic places. Second, a resolution pair is defined to identify a pair of semantic places, which, corresponding to two clauses that can be resolved, can be used to generate a new place in the knowledge Petri net. Such a new place represents an unknown clause implied by the knowledge base. Third, a method is proposed to decide whether a resolution inference is redundant based on the resolution pair. The size of a knowledge Petri net can be reduced, and the inference computation is saved in this way. Fourth, an inference algorithm is proposed employing the resolution pair and knowledge Petri net, proven to be sound and complete. Its computational complexity is polynomial with respect to the number of logical variables and clauses. In addition, another inference algorithm is developed for a given complete knowledge base and a set of newfound clauses. The algorithm is proven to be sound and complete, which offers significantly reduced computational complexity and specifically is linear concerning the number of new clauses. Finally, the wumpus world problem is taken as an example to illustrate and verify the proposed inference algorithms.

Data augmentation and transfer learning for cross-lingual Named Entity Recognition in the biomedical domain

AbstractGiven the increase in production of data for the biomedical field and the unstoppable growth of the internet, the need for Information Extraction (IE) techniques has skyrocketed. Named Entity Recognition (NER) is one of such IE tasks useful for professionals in different areas. There are several settings where biomedical NER is needed, for instance, extraction and analysis of biomedical literature, relation extraction, organisation of biomedical documents, and knowledge-base completion. However, the computational treatment of entities in the biomedical domain has faced a number of challenges including its high cost of annotation, ambiguity, and lack of biomedical NER datasets in languages other than English. These difficulties have hampered data development, affecting both the domain itself and its multilingual coverage. The purpose of this study is to overcome the scarcity of biomedical data for NER in Spanish, for which only two datasets exist, by developing a robust bilingual NER model. Inspired by back-translation, this paper leverages the progress in Neural Machine Translation (NMT) to create a synthetic version of the Colorado Richly Annotated Full-Text (CRAFT) dataset in Spanish. Additionally, a new CRAFT dataset is constructed by replacing 20% of the entities in the original dataset generating a new augmented dataset. We evaluate two training methods: concatenation of datasets and continuous training to assess the transfer learning capabilities of transformers using the newly obtained datasets. The best performing NER system in the development set achieved an F-1 score of 86.39%. The novel methodology proposed in this paper presents the first bilingual NER system and it has the potential to improve applications across under-resourced languages.

Enabling personalised disease diagnosis by combining a patient’s time-specific gene expression profile with a biomedical knowledge base

BackgroundRecent developments in the domain of biomedical knowledge bases (KBs) open up new ways to exploit biomedical knowledge that is available in the form of KBs. Significant work has been done in the direction of biomedical KB creation and KB completion, specifically, those having gene-disease associations and other related entities. However, the use of such biomedical KBs in combination with patients’ temporal clinical data still largely remains unexplored, but has the potential to immensely benefit medical diagnostic decision support systems.ResultsWe propose two new algorithms, LOADDx and SCADDx, to combine a patient’s gene expression data with gene-disease association and other related information available in the form of a KB, to assist personalized disease diagnosis. We have tested both of the algorithms on two KBs and on four real-world gene expression datasets of respiratory viral infection caused by Influenza-like viruses of 19 subtypes. We also compare the performance of proposed algorithms with that of five existing state-of-the-art machine learning algorithms (k-NN, Random Forest, XGBoost, Linear SVM, and SVM with RBF Kernel) using two validation approaches: LOOCV and a single internal validation set. Both SCADDx and LOADDx outperform the existing algorithms when evaluated with both validation approaches. SCADDx is able to detect infections with up to 100% accuracy in the cases of Datasets 2 and 3. Overall, SCADDx and LOADDx are able to detect an infection within 72 h of infection with 91.38% and 92.66% average accuracy respectively considering all four datasets, whereas XGBoost, which performed best among the existing machine learning algorithms, can detect the infection with only 86.43% accuracy on an average.ConclusionsWe demonstrate how our novel idea of using the most and least differentially expressed genes in combination with a KB can enable identification of the diseases that a patient is most likely to have at a particular time, from a KB with thousands of diseases. Moreover, the proposed algorithms can provide a short ranked list of the most likely diseases for each patient along with their most affected genes, and other entities linked with them in the KB, which can support health care professionals in their decision-making.

Pre-training and diagnosing knowledge base completion models

In this work, we introduce and analyze an approach to knowledge transfer from one collection of facts to another without the need for entity or relation matching. The method works for both canonicalized knowledge bases and uncanonicalized or open knowledge bases, i.e., knowledge bases where more than one copy of a real-world entity or relation may exist. The main contribution is a method that can make use of large-scale pre-training on facts, which were collected from unstructured text, to improve predictions on structured data from a specific domain. The introduced method is most impactful on small datasets such as ReVerb20k, where a 6% absolute increase of mean reciprocal rank and 65% relative decrease of mean rank over the previously best method was achieved, despite not relying on large pre-trained models like Bert. To understand the obtained pre-trained models better, we then introduce a novel dataset for the analysis of pre-trained models for Open Knowledge Base Completion, called Doge (Diagnostics of Open knowledge Graph Embeddings). It consists of 6 subsets and is designed to measure multiple properties of a pre-trained model: robustness against synonyms, ability to perform deductive reasoning, presence of gender stereotypes, consistency with reverse relations, and coverage of different areas of general knowledge. Using the introduced dataset, we show that the existing OKBC models lack consistency in presence of synonyms and inverse relations and are unable to perform deductive reasoning. Moreover, their predictions often align with gender stereotypes, which persist even when presented with counterevidence. We additionally investigate the role of pre-trained word embeddings and demonstrate that avoiding biased word embeddings is not a sufficient measure to prevent biased behavior of OKBC models.

Causal lifting and link prediction

Existing causal models for link prediction assume an underlying set of inherent node factors—an innate characteristic defined at the node’s birth—that governs the causal evolution of links in the graph. In some causal tasks, however, link formation is path-dependent : the outcome of link interventions depends on existing links. Unfortunately, these existing causal methods are not designed for path-dependent link formation, as the cascading functional dependencies between links (arising from path dependence ) are either unidentifiable or require an impractical number of control variables. To overcome this, we develop the first causal model capable of dealing with path dependencies in link prediction. In this work, we introduce the concept of causal lifting, an invariance in causal models of independent interest that, on graphs, allows the identification of causal link prediction queries using limited interventional data. Further, we show how structural pairwise embeddings exhibit lower bias and correctly represent the task’s causal structure, as opposed to existing node embeddings, e.g. graph neural network node embeddings and matrix factorization. Finally, we validate our theoretical findings on three scenarios for causal link prediction tasks: knowledge base completion, covariance matrix estimation and consumer-product recommendations.

OpenFact: Factuality Enhanced Open Knowledge Extraction

Abstract We focus on the factuality property during the extraction of an OpenIE corpus named OpenFact, which contains more than 12 million high-quality knowledge triplets. We break down the factuality property into two important aspects—expressiveness and groundedness—and we propose a comprehensive framework to handle both aspects. To enhance expressiveness, we formulate each knowledge piece in OpenFact based on a semantic frame. We also design templates, extra constraints, and adopt human efforts so that most OpenFact triplets contain enough details. For groundedness, we require the main arguments of each triplet to contain linked Wikidata1 entities. A human evaluation suggests that the OpenFact triplets are much more accurate and contain denser information compared to OPIEC-Linked (Gashteovski et al., 2019), one recent high-quality OpenIE corpus grounded to Wikidata. Further experiments on knowledge base completion and knowledge base question answering show the effectiveness of OpenFact over OPIEC-Linked as supplementary knowledge to Wikidata as the major KG.

A Proposal for Basic Formal Ontology for Knowledge Management in Building Information Modeling Domain

The construction industry produces enormous amounts of information, relying on building information modeling (BIM). However, due to interoperability issues, valuable information is not being used properly. Ontology offers a solution to this interoperability. A complete knowledge base can be provided by reusing basic formal ontology (BFO). In previous studies, domain ontology was developed without BFO. Domain ontology requires loads of effort to reuse because domain ontology is too detailed. To increase the reuse rate and establish a complete knowledge base, it is necessary to develop BFO. This study has developed the BFO in the BIM domain to advance interoperability. First, unnecessary parts were omitted from the existing BFO development process, the process was simplified, and the base of hierarchy was created by extracting the most basic superclasses of the BFO model from Revit, the software of BIM. Based on that hierarchy, each child class was created, and the BFO model was completed by completing the relation of each class. After completion of the model, reliability, in addition to the completeness of the model, was evaluated through a query. Domain experts can reuse the BFO when defining relations between concepts and entities. The proposed BFO will be the foundation of future ontology developments in the BIM domain. This study facilitates future researchers to enhance interoperability in the BIM domain and make the ontology more complete to improve information sharing.

A prefix and attention map discrimination fusion guided attention for biomedical named entity recognition

BackgroundThe biomedical literature is growing rapidly, and it is increasingly important to extract meaningful information from the vast amount of literature. Biomedical named entity recognition (BioNER) is one of the key and fundamental tasks in biomedical text mining. It also acts as a primitive step for many downstream applications such as relation extraction and knowledge base completion. Therefore, the accurate identification of entities in biomedical literature has certain research value. However, this task is challenging due to the insufficiency of sequence labeling and the lack of large-scale labeled training data and domain knowledge.ResultsIn this paper, we use a novel word-pair classification method, design a simple attention mechanism and propose a novel architecture to solve the research difficulties of BioNER more efficiently without leveraging any external knowledge. Specifically, we break down the limitations of sequence labeling-based approaches by predicting the relationship between word pairs. Based on this, we enhance the pre-trained model BioBERT, through the proposed prefix and attention map dscrimination fusion guided attention and propose the E-BioBERT. Our proposed attention differentiates the distribution of different heads in different layers in the BioBERT, which enriches the diversity of self-attention. Our model is superior to state-of-the-art compared models on five available datasets: BC4CHEMD, BC2GM, BC5CDR-Disease, BC5CDR-Chem, and NCBI-Disease, achieving F1-score of 92.55%, 85.45%, 87.53%, 94.16% and 90.55%, respectively.ConclusionCompared with many previous various models, our method does not require additional training datasets, external knowledge, and complex training process. The experimental results on five BioNER benchmark datasets demonstrate that our model is better at mining semantic information, alleviating the problem of label inconsistency, and has higher entity recognition ability. More importantly, we analyze and demonstrate the effectiveness of our proposed attention.

HyperQuaternionE: A hyperbolic embedding model for qualitative spatial and temporal reasoning

Qualitative spatial/temporal reasoning (QSR/QTR) plays a key role in research on human cognition, e.g., as it relates to navigation, as well as in work on robotics and artificial intelligence. Although previous work has mainly focused on various spatial and temporal calculi, more recently representation learning techniques such as embedding have been applied to reasoning and inference tasks such as query answering and knowledge base completion. These subsymbolic and learnable representations are well suited for handling noise and efficiency problems that plagued prior work. However, applying embedding techniques to spatial and temporal reasoning has received little attention to date. In this paper, we explore two research questions: (1) How do embedding-based methods perform empirically compared to traditional reasoning methods on QSR/QTR problems? (2) If the embedding-based methods are better, what causes this superiority? In order to answer these questions, we first propose a hyperbolic embedding model, called HyperQuaternionE, to capture varying properties of relations (such as symmetry and anti-symmetry), to learn inversion relations and relation compositions (i.e., composition tables), and to model hierarchical structures over entities induced by transitive relations. We conduct various experiments on two synthetic datasets to demonstrate the advantages of our proposed embedding-based method against existing embedding models as well as traditional reasoners with respect to entity inference and relation inference. Additionally, our qualitative analysis reveals that our method is able to learn conceptual neighborhoods implicitly. We conclude that the success of our method is attributed to its ability to model composition tables and learn conceptual neighbors, which are among the core building blocks of QSR/QTR.

Graph‐based deep learning model for knowledge base completion in constraint management of construction projects

AbstractConstruction projects face various constraints in terms of materials, labor, equipment, and documents, which can interrupt the scheduled work. Package‐based constraint management (PCM) is a state‐of‐the‐art graph‐based approach that follows the lean theory to effectively model, monitor, and remove constraints before the commencement of work, ensuring smooth construction and minimizing delay and waste. PCM relies on exploring and investigating project knowledge bases (KBs), formed by entity‐relation‐entity triples of constraints. However, most PCM KBs are incomplete and suffer from poor semantics, which hinders the PCM functions. Although many KB completion (KBC) methods exist in the field of artificial intelligence, they primarily focus on general knowledge and exclude the features of specific domains. Therefore, they cannot be directly applied to complete PCM KBs. To address the issue, this study proposes a novel deep learning model, referred to as the domain information enhanced graph neural network (D‐GNN). The features of the developed D‐GNN include (1) building a domain ontology to enrich semantics with rule reasoning, (2) applying the GNN to learn and encode embeddings of constraint entities and relations, and (3) employing a convolution neural network (CNN) for decoding and identifying missing triples. D‐GNN improves the existing KBC methods by integrating two types of domain information, namely, the ontological classes and working contexts into GNN and CNN, respectively. The experimental results verified that the D‐GNN reached an accuracy of 0.848–0.951, and the domain information integration increased the performance by up to 0.263. In practical testing, the D‐GNN significantly reduced the KBC time to 1/6–1/35 of the manual approach and reached higher accuracy. Therefore, the proposed D‐GNN can facilitate PCM by providing complete KBs and supporting downstream constraint monitoring and removal.

Unifying Knowledge Base Completion with PU Learning to Mitigate the Observation Bias

Methods for Knowledge Base Completion (KBC) reason about a knowledge base (KB) in order to derive new facts that should be included in the KB. This is challenging for two reasons. First, KBs only contain positive examples. This complicates model evaluation which needs both positive and negative examples. Second, those facts that were selected to be included in the knowledge base, are most likely not an i.i.d. sample of the true facts, due to the way knowledge bases are constructed. In this paper, we focus on rule-based approaches, which traditionally address the first challenge by making assumptions that enable identifying negative examples, which in turn makes it possible to compute a rule's confidence or precision. However, they largely ignore the second challenge, which means that their estimates of a rule's confidence can be biased. This paper approaches rule-based KBC through the lens of PU-learning, which can cope with both challenges. We make three contributions.: (1) We provide a unifying view that formalizes the relationship between multiple existing confidences measures based on (i) what assumption they make about and (ii) how their accuracy depends on the selection mechanism. (2) We introduce two new confidence measures that can mitigate known biases by using propensity scores that quantify how likely a fact is to be included the KB. (3) We show through theoretical and empirical analysis that taking the bias into account improves the confidence estimates, even when the propensity scores are not known exactly.

KGBoost: A classification-based knowledge base completion method with negative sampling

Knowledge base completion is formulated as a binary classification problem in this work, where an XGBoost binary classifier is trained for each relation using relevant links in knowledge graphs (KGs). The new method, named KGBoost, adopts a modularized design and attempts to find hard negative samples so as to train a powerful classifier for missing link prediction. We conduct experiments on multiple benchmark datasets, and demonstrate that KGBoost outperforms state-of-the-art methods across most datasets. Furthermore, as compared with models trained by end-to-end optimization, KGBoost works well under the low-dimensional setting so as to allow a smaller model size.

A Dynamic Convolutional Network-Based Model for Knowledge Graph Completion

Knowledge graph embedding can learn low-rank vector representations for knowledge graph entities and relations, and has been a main research topic for knowledge graph completion. Several recent works suggest that convolutional neural network (CNN)-based models can capture interactions between head and relation embeddings, and hence perform well on knowledge graph completion. However, previous convolutional network models have ignored the different contributions of different interaction features to the experimental results. In this paper, we propose a novel embedding model named DyConvNE for knowledge base completion. Our model DyConvNE uses a dynamic convolution kernel because the dynamic convolutional kernel can assign weights of varying importance to interaction features. We also propose a new method of negative sampling, which mines hard negative samples as additional negative samples for training. We have performed experiments on the data sets WN18RR and FB15k-237, and the results show that our method is better than several other benchmark algorithms for knowledge graph completion. In addition, we used a new test method when predicting the Hits@1 values of WN18RR and FB15k-237, named specific-relationship testing. This method gives about a 2% relative improvement over models that do not use this method in terms of Hits@1.

Bridging symbolic and subsymbolic reasoning with minimax entropy models

In this paper, we investigate MiniMax Entropy models, a class of neural symbolic models where symbolic and subsymbolic features are seamlessly integrated. We show how these models recover classical algorithms from both the deep learning and statistical relational learning scenarios. Novel hybrid settings are defined and experimentally explored, showing state-of-the-art performance in collective classification, knowledge base completion and graph (molecular) data generation.

Association Rules Enhanced Knowledge Graph Attention Network

Embedding knowledge graphs into continuous vector spaces has recently attracted increasing interest in knowledge base completion. However, in most existing embedding methods, only fact triplets are utilized, and logical rules have not been thoroughly studied for the knowledge base completion task. To overcome the problem, we propose an association rules enhanced knowledge graph attention network (AR-KGAN). In this paper, triplets and logical rules are jointly modeled in the proposed unified framework to achieve more predictive entity and relation embeddings. Association rules and corresponding correlation degrees between them can be automatically obtained according to our designed mining algorithm. The major component of AR-KGAN is an encoder of an effective neighborhood aggregator, which addresses the problems by aggregating neighbors with both association rules based and graph based attention weights. The decoder enables AR-KGAN to be translational between entities and relations while keeping the superior link prediction performance. Then, the global loss is minimized over both atomic and complex formulas to achieve the embedding task. In this manner, we learn embeddings compatible with triplets and association rules, which are certainly more predictive for knowledge acquisition and inference. The results show that the proposed AR-KGAN model achieves significant and consistent improvements over state-of-the-art methods on three benchmark datasets.

Siamese Pre-Trained Transformer Encoder for Knowledge Base Completion

Siamese Pre-Trained Transformer Encoder for Knowledge Base Completion

Predicting affinity ties in a surname network.

From administrative registers of last names in Santiago, Chile, we create a surname affinity network that encodes socioeconomic data. This network is a multi-relational graph with nodes representing surnames and edges representing the prevalence of interactions between surnames by socioeconomic decile. We model the prediction of links as a knowledge base completion problem, and find that sharing neighbors is highly predictive of the formation of new links. Importantly, We distinguish between grounded neighbors and neighbors in the embedding space, and find that the latter is more predictive of tie formation. The paper discusses the implications of this finding in explaining the high levels of elite endogamy in Santiago.

Populating Web-Scale Knowledge Graphs Using Distantly Supervised Relation Extraction and Validation

In this paper, we propose a fully automated system to extend knowledge graphs using external information from web-scale corpora. The designed system leverages a deep-learning-based technology for relation extraction that can be trained by a distantly supervised approach. In addition, the system uses a deep learning approach for knowledge base completion by utilizing the global structure information of the induced KG to further refine the confidence of the newly discovered relations. The designed system does not require any effort for adaptation to new languages and domains as it does not use any hand-labeled data, NLP analytics, and inference rules. Our experiments, performed on a popular academic benchmark, demonstrate that the suggested system boosts the performance of relation extraction by a wide margin, reporting error reductions of 50%, resulting in relative improvement of up to 100%. Furthermore, a web-scale experiment conducted to extend DBPedia with knowledge from Common Crawl shows that our system is not only scalable but also does not require any adaptation cost, while yielding a substantial accuracy gain.

Taylor’s theorem: A new perspective for neural tensor networks

Neural tensor networks have been widely used in a large number of natural language processing tasks such as conversational sentiment analysis, named entity recognition and knowledge base completion. However, the mathematical explanation of neural tensor networks remains a challenging problem, due to the bilinear term. According to Taylor’s theorem, a kth order differentiable function can be approximated by a kth order Taylor polynomial around a given point. Therefore, we provide a mathematical explanation of neural tensor networks and also reveal the inner link between them and feedforward neural networks from the perspective of Taylor’s theorem. In addition, we unify two forms of neural tensor networks into a single framework and present factorization methods to make the neural tensor networks parameter-efficient. Experimental results bring some valuable insights into neural tensor networks.

A structure distinguishable graph attention network for knowledge base completion

A structure distinguishable graph attention network for knowledge base completion