Abstract
Methods based on representation learning currently hold the state-of-the-art in many natural language processing and knowledge base inference tasks. Yet, a major challenge is how to efficiently incorporate commonsense knowledge into such models. A recent approach regularizes relation and entity representations by propositionalization of first-order logic rules. However, propositionalization does not scale beyond domains with only few entities and rules. In this paper we present a highly efficient method for incorporating implication rules into distributed representations for automated knowledge base construction. We map entity-tuple embeddings into an approximately Boolean space and encourage a partial ordering over relation embeddings based on implication rules mined from WordNet. Surprisingly, we find that the strong restriction of the entity-tuple embedding space does not hurt the expressiveness of the model and even acts as a regularizer that improves generalization. By incorporating few commonsense rules, we achieve an increase of 2 percentage points mean average precision over a matrix factorization baseline, while observing a negligible increase in runtime.
Highlights
Current successful methods for automated knowledge base construction tasks heavily rely on learned distributed vector representations (Nickel et al, 2012; Riedel et al, 2013; Socher et al, 2013; Chang et al, 2014; Neelakantan et al, 2015; Toutanova et al, 2015; Nickel et al, 2015; Verga et al, 2016; Verga and McCallum, 2016)
Every first-order rule is propositionalized based on observed entity-tuples, and a differentiable loss term is added for every propositional rule
Our contributions are fourfold: (i) we develop a very efficient way of regularizing relation representations to incorporate first-order logic implications (§3), (ii) we reveal that, against expectation, mapping entity-tuple embeddings to non-negative space does not hurt but instead improves the generalization ability of our model (§5.1) (iii) we show improvements on a knowledge base completion task by injecting mined commonsense rules from WordNet (§5.3), and (iv) we give a qualitative analysis of the results, demonstrating that implication constraints are satisfied in an asymmetric way and result in a substantially increased structuring of the relation embedding space (§5.6)
Summary
Current successful methods for automated knowledge base construction tasks heavily rely on learned distributed vector representations (Nickel et al, 2012; Riedel et al, 2013; Socher et al, 2013; Chang et al, 2014; Neelakantan et al, 2015; Toutanova et al, 2015; Nickel et al, 2015; Verga et al, 2016; Verga and McCallum, 2016) These models are able to learn robust representations from large amounts of data, they often lack commonsense knowledge. A recent approach (Rocktaschel et al, 2015) regularizes entity-tuple and relation embeddings via first-order logic rules To this end, every first-order rule is propositionalized based on observed entity-tuples, and a differentiable loss term is added for every propositional rule. Every entity-tuple t ∈ T is represented by a latent vector t ∈ Rk (with T the set of all entity-tuples in O)
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
