Knowledge Base Completion Research Articles

Scene Graph Prediction with Limited Labels.

Visual knowledge bases such as Visual Genome power numerous applications in computer vision, including visual question answering and captioning, but suffer from sparse, incomplete relationships. All scene graph models to date are limited to training on a small set of visual relationships that have thousands of training labels each. Hiring human annotators is expensive, and using textual knowledge base completion methods are incompatible with visual data. In this paper, we introduce a semi-supervised method that assigns probabilistic relationship labels to a large number of unlabeled images using few' labeled examples. We analyze visual relationships to suggest two types of image-agnostic features that are used to generate noisy heuristics, whose outputs are aggregated using a factor graph-based generative model. With as few as 10 labeled examples per relationship, the generative model creates enough training data to train any existing state-of-the-art scene graph model. We demonstrate that our method outperforms all baseline approaches on scene graph prediction by 5.16 recall@ 100 for PREDCLS. In our limited label setting, we define a complexity metric for relationships that serves as an indicator (R2 = 0.778) for conditions under which our method succeeds over transfer learning, the de-facto approach for training with limited labels.

A convolutional neural network-based model for knowledge base completion and its application to search personalization

In this paper, we propose a novel embedding model, named ConvKB, for knowledge base completion. Our model ConvKB advances state-of-the-art models by employing a convolutional neural network, so that it can capture global relationships and transitional characteristics between entities and relations in knowledge bases. In ConvKB, each triple (head entity, relation, tail entity) is represented as a 3-column matrix where each column vector represents a triple element. This 3-column matrix is then fed to a convolution layer where multiple filters are operated on the matrix to generate different feature maps. These feature maps are then concatenated into a single feature vector representing the input triple. The feature vector is multiplied with a weight vector via a dot product to return a score. This score is then used to predict whether the triple is valid or not. Experiments show that ConvKB obtains better link prediction and triple classification results than previous state-of-the-art models on benchmark datasets WN18RR, FB15k-237, WN11 and FB13. We further apply our ConvKB to a search personalization problem which aims to tailor the search results to each specific user based on the user’s personal interests and preferences. In particular, we model the potential relationship between the submitted query, the user and the search result (i.e., document) as a triple (query, user, document) on which the ConvKB is able to work. Experimental results on query logs from a commercial web search engine show that ConvKB achieves better performances than the standard ranker as well as strong search personalization baselines.

On Completing Sparse Knowledge Base with Transitive Relation Embedding

Multi-relation embedding is a popular approach to knowledge base completion that learns embedding representations of entities and relations to compute the plausibility of missing triplet. The effectiveness of embedding approach depends on the sparsity of KB and falls for infrequent entities that only appeared a few times. This paper addresses this issue by proposing a new model exploiting the entity-independent transitive relation patterns, namely Transitive Relation Embedding (TRE). The TRE model alleviates the sparsity problem for predicting on infrequent entities while enjoys the generalisation power of embedding. Experiments on three public datasets against seven baselines showed the merits of TRE in terms of knowledge base completion accuracy as well as computational complexity.

Combining Axiom Injection and Knowledge Base Completion for Efficient Natural Language Inference

In logic-based approaches to reasoning tasks such as Recognizing Textual Entailment (RTE), it is important for a system to have a large amount of knowledge data. However, there is a tradeoff between adding more knowledge data for improved RTE performance and maintaining an efficient RTE system, as such a big database is problematic in terms of the memory usage and computational complexity. In this work, we show the processing time of a state-of-the-art logic-based RTE system can be significantly reduced by replacing its search-based axiom injection (abduction) mechanism by that based on Knowledge Base Completion (KBC). We integrate this mechanism in a Coq plugin that provides a proof automation tactic for natural language inference. Additionally, we show empirically that adding new knowledge data contributes to better RTE performance while not harming the processing speed in this framework.

End-to-End Structure-Aware Convolutional Networks for Knowledge Base Completion.

Knowledge graph embedding has been an active research topic for knowledge base completion, with progressive improvement from the initial TransE, TransH, DistMult et al to the current state-of-the-art ConvE. ConvE uses 2D convolution over embeddings and multiple layers of nonlinear features to model knowledge graphs. The model can be efficiently trained and scalable to large knowledge graphs. However, there is no structure enforcement in the embedding space of ConvE. The recent graph convolutional network (GCN) provides another way of learning graph node embedding by successfully utilizing graph connectivity structure. In this work, we propose a novel end-to-end Structure-Aware Convolutional Network (SACN) that takes the benefit of GCN and ConvE together. SACN consists of an encoder of a weighted graph convolutional network (WGCN), and a decoder of a convolutional network called Conv-TransE. WGCN utilizes knowledge graph node structure, node attributes and edge relation types. It has learnable weights that adapt the amount of information from neighbors used in local aggregation, leading to more accurate embeddings of graph nodes. Node attributes in the graph are represented as additional nodes in the WGCN. The decoder Conv-TransE enables the state-of-the-art ConvE to be translational between entities and relations while keeps the same link prediction performance as ConvE. We demonstrate the effectiveness of the proposed SACN on standard FB15k-237 and WN18RR datasets, and it gives about 10% relative improvement over the state-of-the-art ConvE in terms of HITS@1, HITS@3 and HITS@10.

LENA: Locality-Expanded Neural Embedding for Knowledge Base Completion

Embedding based models for knowledge base completion have demonstrated great successes and attracted significant research interest. In this work, we observe that existing embedding models all have their loss functions decomposed into atomic loss functions, each on a triple or an postulated edge in the knowledge graph. Such an approach essentially implies that conditioned on the embeddings of the triple, whether the triple is factual is independent of the structure of the knowledge graph. Although arguably the embeddings of the entities and relation in the triple contain certain structural information of the knowledge base, we believe that the global information contained in the embeddings of the triple can be insufficient and such an assumption is overly optimistic in heterogeneous knowledge bases. Motivated by this understanding, in this work we propose a new embedding model in which we discard the assumption that the embeddings of the entities and relation in a triple is a sufficient statistic for the triple’s factual existence. More specifically, the proposed model assumes that whether a triple is factual depends not only on the embedding of the triple but also on the embeddings of the entities and relations in a larger graph neighbourhood. In this model, attention mechanisms are constructed to select the relevant information in the graph neighbourhood so that irrelevant signals in the neighbourhood are suppressed. Termed locality-expanded neural embedding with attention (LENA), this model is tested on four standard datasets and compared with several stateof-the-art models for knowledge base completion. Extensive experiments suggest that LENA outperforms the existing models in virtually every metric.

Distribution-Aware Crowdsourced Entity Collection

The problem of crowdsourced entity collection solicits people (a.k.a. workers) to complete missing data in a database and has witnessed many applications in knowledge base completion and enterprise data collection. Although previous studies have attempted to address the “open world” challenge of crowdsourced entity collection, they do not pay much attention to the “distribution” of the collected entities. Evidently, in many real applications, users may have distribution requirements on the collected entities, e.g., even spatial distribution when collecting points-of-interest. In this paper, we study a new research problem, distribution-aware crowdsourced entity collection (CrowdDEC): Given an expected distribution w.r.t. an attribute (e.g., region or year), it aims to collect a set of entities via crowdsourcing and minimize the difference of the entity distribution from the expected distribution. Due to the openness of crowdsourcing, the CrowdDEC problem calls for effective crowdsourcing quality control. We propose an adaptive worker selection approach to address this problem. The approach estimates underlying entity distribution of workers on-the-fly based on the collected entities. Then, it adaptively selects the best set of workers that minimizes the difference from the expected distribution. Once workers submit their answers, it adjusts the estimation of workers' underlying distributions for subsequent adaptive worker selections. We prove the hardness of the problem, and develop effective estimation techniques as well as efficient worker selection algorithms to support this approach. We deployed the proposed approach on Amazon Mechanical Turk and the experimental results on two real datasets show that the approach achieves superiority on both effectiveness and efficiency.

Seasonality overwhelms aquacultural activity in determining the composition and assembly of the bacterial community in Lake Taihu, China

Aquaculture accounts for an extremely valuable and rapidly expanding sector of global food production, yet its environmental impacts on aquatic ecosystems have attracted much concern. In this study, we collected water samples from eastern Lake Taihu, China. We targeted sites varying in their intensity of aquacultural activities and sampled them over multiple seasons. For each sample, we measured physicochemical variables, and we sequenced the 16S rRNA gene of the respective bacterial communities using an Illumina second-generation sequencing platform. Marked differences in diversity and bacterial community composition were observed between seasons, whereas we observed relatively weak differences between sites. Remarkable differences in the abundance of the bacterial community were observed at the phylum and genus levels across the different seasons. Stochastic processes dominated the assembly of bacterial communities in the aquaculture-influenced systems, and the assembly processes of bacterial community differed between seasons. Our observations highlight the effect of seasonality on bacterial communities and provide a more complete knowledge base for the proper assessment of the effects of aquacultural activities on freshwater ecosystems.

ByNowLife: A Novel Framework for OWL and Bayesian Network Integration

An ontology-based system can currently logically reason through the Web Ontology Language Description Logic (OWL DL). To perform probabilistic reasoning, the system must use a separate knowledge base, separate processing, or third-party applications. Previous studies mainly focus on how to represent probabilistic information in ontologies and perform reasoning through them. These approaches are not suitable for systems that already have running ontologies and Bayesian network (BN) knowledge bases because users must rewrite the probabilistic information contained in a BN into an ontology. We present a framework called ByNowLife, which is a novel approach for integrating BN with OWL by providing an interface for retrieving probabilistic information through SPARQL queries. ByNowLife catalyzes the integration process by transforming logical information contained in an ontology into a BN and probabilistic information contained in a BN into an ontology. This produces a system with a complete knowledge base. Using ByNowLife, a system that already has separate ontologies and BN knowledge bases can integrate them into a single knowledge base and perform both logical and probabilistic reasoning through it. The integration not only facilitates the unity of reasoning but also has several other advantages, such as ontology enrichment and BN structural adjustment through structural and parameter learning.

XLORE2: Large-scale Cross-lingual Knowledge Graph Construction and Application

Knowledge bases (KBs) are often greatly incomplete, necessitating a demand for KB completion. Although XLORE is an English-Chinese bilingual knowledge graph, there are only 423,974 cross-lingual links between English instances and Chinese instances. We present XLORE2, an extension of the XLORE that is built automatically from Wikipedia, Baidu Baike and Hudong Baike. We add more facts by making cross-lingual knowledge linking, cross-lingual property matching and fine-grained type inference. We also design an entity linking system to demonstrate the effectiveness and broad coverage of XLORE2.

Fuzzy Knowledge Inference: Quickly Estimate Evidence via Formula Embedding

Inference on Knowledge Bases (KBs) is an important way to construct more complete KBs and answer KB questions. Inference can be viewed as a process from evidence to conclusion following specific formulas. Traditional methods usually search on the KB to collect evidence, which cannot apply to large-scale KBs, because the running time of searching increases radically as the scale of KBs increases. What is worse, evidence cannot be found if one fact in it is missing, which may result in the failure of inference. To this end, we propose a fuzzy method of estimating evidence, which replaces searching by estimating the existence of evidence by constructing formula embeddings, and then we merge these estimations into a probabilistic model to infer conclusions. This method can apply to large-scale KBs, because estimating evidence is very fast and is irrelevant to the KB scale. Estimating evidence can also be viewed as fuzzy matching, so this method can handle the situation where facts are missing. We evaluate this method on the knowledge base completion task, and it achieves a better performance than state-of-the-art methods and has a shorter running time.

A Comparison of Word Embeddings and N-gram Models for DBpedia Type and Invalid Entity Detection

This article presents and evaluates a method for the detection of DBpedia types and entities that can be used for knowledge base completion and maintenance. This method compares entity embeddings with traditional N-gram models coupled with clustering and classification. We tackle two challenges: (a) the detection of entity types, which can be used to detect invalid DBpedia types and assign DBpedia types for type-less entities; and (b) the detection of invalid entities in the resource description of a DBpedia entity. Our results show that entity embeddings outperform n-gram models for type and entity detection and can contribute to the improvement of DBpedia’s quality, maintenance, and evolution.

Knowledge Base Completion by Variational Bayesian Neural Tensor Decomposition

Knowledge base completion is an important research problem in knowledge bases, which play important roles in question answering, information retrieval, and other applications. A number of relational learning algorithms have been proposed to solve this problem. However, despite their success in modeling the entity relations, they are not well founded in a Bayesian manner and thus are hard to model the prior information of the entity and relation factors. Furthermore, they under-represent the interaction between entity and relation factors. In order to avoid these disadvantages, we provide a neural-inspired approach, namely Bayesian Neural Tensor Decomposition approach for knowledge base completion based on the Stochastic Gradient Variational Bayesian framework. We employ a multivariate Bernoulli likelihood function to represent the existence of facts in knowledge graphs. We further employ a Multi-layered Perceptrons to represent more complex interactions between the latent subject, predicate, and object factors. The SGVB framework can enable us to make efficient approximate variational inference for the proposed nonlinear probabilistic tensor decomposition by a novel local reparameterization trick. This way avoids the need of expensive iterative inference schemes such as MCMC and does not make any over-simplified assumptions about the posterior distributions, in contrary to the common variational inference. In order to evaluate the proposed model, we have conducted experiments on real-world knowledge bases, i.e., FreeBase and WordNet. Experimental results have indicated the promising performance of the proposed method.

Estimating the Class Prior in Positive and Unlabeled Data Through Decision Tree Induction

For tasks such as medical diagnosis and knowledge base completion, a classifier may only have access to positive and unlabeled examples, where the unlabeled data consists of both positive and negative examples. One way that enables learning from this type of data is knowing the true class prior. In this paper, we propose a simple yet effective method for estimating the class prior, by estimating the probability that a positive example is selected to be labeled. Our key insight is that subdomains of the data give a lower bound on this probability. This lower bound gets closer to the real probability as the ratio of labeled examples increases. Finding such subsets can naturally be done via top-down decision tree induction. Experiments show that our method makes estimates which are equivalently accurate as those of the state of the art methods, and is an order of magnitude faster.

Data-Dependent Learning of Symmetric/Antisymmetric Relations for Knowledge Base Completion

Embedding-based methods for knowledge base completion (KBC) learn representations of entities and relations in a vector space, along with the scoring function to estimate the likelihood of relations between entities. The learnable class of scoring functions is designed to be expressive enough to cover a variety of real-world relations, but this expressive comes at the cost of an increased number of parameters. In particular, parameters in these methods are superfluous for relations that are either symmetric or antisymmetric. To mitigate this problem, we propose a new L1 regularizer for Complex Embeddings, which is one of the state-of-the-art embedding-based methods for KBC. This regularizer promotes symmetry or antisymmetry of the scoring function on a relation-by-relation basis, in accordance with the observed data. Our empirical evaluation shows that the proposed method outperforms the original Complex Embeddings and other baseline methods on the FB15k dataset.

Corpus-Level Fine-Grained Entity Typing

Extracting information about entities remains an important research area. This paper addresses the problem of corpus-level entity typing, i.e., inferring from a large corpus that an entity is a member of a class, such as "food" or "artist". The application of entity typing we are interested in is knowledge base completion, specifically, to learn which classes an entity is a member of. We propose FIGMENT to tackle this problem. FIGMENT is embedding-based and combines (i) a global model that computes scores based on global information of an entity and (ii) a context model that first evaluates the individual occurrences of an entity and then aggregates the scores. Each of the two proposed models has specific properties. For the global model, learning high-quality entity representations is crucial because it is the only source used for the predictions. Therefore, we introduce representations using the name and contexts of entities on the three levels of entity, word, and character. We show that each level provides complementary information and a multi-level representation performs best. For the context model, we need to use distant supervision since there are no context-level labels available for entities. Distantly supervised labels are noisy and this harms the performance of models. Therefore, we introduce and apply new algorithms for noise mitigation using multi-instance learning. We show the effectiveness of our models on a large entity typing dataset built from Freebase.

A Knowledge Base Completion Model Based on Path Feature Learning.

Large-scale knowledge bases, as the foundations for promoting the development of artificial intelligence, have attracted increasing attention in recent years. These knowledge bases contain billions of facts in triple format; yet, they suffer from sparse relations between entities. Researchers proposed the path ranking algorithm (PRA) to solve this fatal problem. To improve the scalability of knowledge inference, PRA exploits random walks to find Horn clauses with chain structures to predict new relations given existing facts. This method can be regarded as a statistical classification issue for statistical relational learning (SRL). However, large-scale knowledge base completion demands superior accuracy and scalability. In this paper, we propose the path feature learning model (PFLM) to achieve this urgent task. More precisely, we define a two-stage model: the first stage aims to learn path features from the existing knowledge base and extra parsed corpus; the second stage uses these path features to predict new relations. The experimental results demonstrate that the PFLM can learn meaningful features and can achieve significant and consistent improvements compared with previous work.

Knowledge Base Completion with Out-of-Knowledge-Base Entities: A Graph Neural Network Approach

Knowledge base completion (KBC) aims to predict missing information in a knowledge base.In this paper, we address the out-of-knowledge-base (OOKB) entity problem in KBC:how to answer queries concerning test entities not observed at training time. Existing embedding-based KBC models assume that all test entities are available at training time, making it unclear how to obtain embeddings for new entities without costly retraining. To solve the OOKB entity problem without retraining, we use graph neural networks (Graph-NNs) to compute the embeddings of OOKB entities, exploiting the limited auxiliary knowledge provided at test time.The experimental results show the effectiveness of our proposed model in the OOKB setting.Additionally, in the standard KBC setting in which OOKB entities are not involved, our model achieves state-of-the-art performance on the WordNet dataset. The code and dataset are available at https://github.com/takuo-h/GNN-for-OOKB

Constructing a knowledge base for entity linking on Irish cultural heritage collections

A known problem for proponents of Entity Linking in Cultural Heritage collections is the poor representation of entities in popular knowledge bases such as DBpedia. This is largely due to the niche nature of the entities contained in these collections. Where such problems arise it may be possible to generate more complete knowledge bases from existing resources used by scholars who specialise in the domain of the collection in question. This paper presents the process by which a knowledge base for informing an entity linker about people in Irish cultural heritage collections was developed from the Dictionary of Irish Biography, a compendium of biographies about notable Irish figures written by Irish historians. We present the design considerations which influenced the structure of the knowledge base given its intended application and the process by which the desired information was extracted from the collection of biographies. This includes the description of an automatic linking process for associating entities in the new knowledge base with their corresponding entries in DBpedia where such a correspondence was possible. We discuss other resources which possess similar properties to the Dictionary of Irish Biography that may be useful for expanding the current knowledge base. A preliminary test on a collection of Irish manuscripts is conducted using an existing Entity Linking tool to demonstrate the use of this knowledge base in a linking task.

Extraction of fuzzy rules at different concept levels related to image features of mammography for diagnosis of breast cancer

Mammography is an inexpensive and non-invasive method through which one can diagnose breast cancer in its early stages. As these images need interpretation by a radiologist, this may develop some problems due to fatigue, repetition, and need for a great deal of attention to details and other factors. Thus, a method capable of diagnosing breast cancer should be employed to help physicians in this regard.In this paper, The mini Mammographic Image Analysis Society (mini-MIAS) database of mammograms is used. The aim is to distinguish between normal and abnormal classes. In the preprocessing stage, noise removal, removal of labels of images, heightening the contrast, and ROI segmentation are performed, and then compactness, entropy, mean, and smoothness are extracted from the images. In addition to classification, we have come to a new approach in order to create a complete knowledge base, which then we use this knowledge base for classification. We have a comprehensive knowledge base which covers all the conceptual levels.The extracted features are referred to as fuzzy classifiers through the look-up table method. And, for evaluation of the results, the 10-fold method is used. Discretization operations are performed on training data across 2, 3, and 4 levels to develop concept hierarchy. Concept hierarchies reduce the data by replacing low-level concepts with higher-level concepts and the outcome is more meaningful and easier to interpret. Eventually, Bagging algorithm is used for finding out the majority vote and the final result of the discretization levels. The obtained accuracy is 89.37±6.62.