Lexical Entailment Research Articles

Flexible margins and multiple samples learning to enhance lexical semantic similarity

The advancement of deep learning and neural networks has led to the widespread adoption of neural word embeddings as a prominent lexical representation method in natural language processing. With the help of the neural language model trained by the contextual information of large scale text, the neural word embedding obtained by the neural language model captures more semantic correlation in the semantic space, while ignoring the semantic similarity. It will incur high computational cost and time costs during the training process of the model. To better inject semantic similarity into the distribution space and reduce time cost, we perform post processing learning of neural word embeddings using deep metric learning. This paper proposes a lexical enhancement method based on flexible margins and multiple samples learning. In this method, we embed the lexical entailment constraint relations into neural word embeddings. By categorizing the set of lexical constraints and penalizing the negative samples to different degrees according to the gap between categories, and allowing the positive and negative samples to learn from each other in the distributed space. The method we propose significantly improves neural word embeddings. By evaluating neural word embedded vocabulary similarity, the benchmark accuracy is improved to 75%. The method shows great competitiveness in text similarity tasks and text categorization tasks. These findings summarize research results and provide strong support for further applications.

Semantic Inheritance in the Lexical Paradigms of Old English Strong Verbs

This work delves into the semantics of Old English lexical paradigms based on strong verbs. Its aim is to describe the patterns of semantic inheritance that hold in these paradigms, which present morphologically related words sharing the form and meaning of the base of derivation. The analysis carried out permits to circumscribe semantic derivation in these Old English lexical paradigms into the lexical entailment relations of troponymy, -troponymy, backward presupposition and cause, and the semantic relations of synonymy and opposition. The data of research has been retrieved from the lexical database Nerthus (Martín Arista et. al 2016). On the theoretical side, this examination follows the English lexical database WordNet (Princeton, 2010). This research unfolds a systematic methodology that is thoroughly described and illustrated by means of the paradigm (ge)berstan. It has not only been possible to circumscribe semantic derivation into the six semantic relations mentioned, but also to determine their frequency of occurrence in the paradigms under analysis, which indicates that synonymy and troponymy are the most recurrent semantic relations.

The Semantic Map of "Aktionart" and Lexical Entailment of Old English Strong Verbs

The aim of this article is to analyse the semantic relations that hold between Old English primitive and derived verbs in terms of troponymy and Aktionsart. The results of this analysis are presented in a semantic map, while emphasis is made on the points of contact between these phenomena. The main conclusion is that semantic maps represent a more flexible and applicable methodology than previous work suggests since they have been used to deal with one language, to explain historical languages and to refer to specific lexical items. Likewise, this analysis shows evidence of an inherent relationship between both phenomena: troponymy and Aktionsart.

Combining Specialized Word Embeddings and Subword Semantic Features for Lexical Entailment Recognition

The challenge of Lexical Entailment Recognition (LER) aims to identify the is-a relation between words. This problem has recently received attention from researchers in the field of natural language processing because of its application to varied downstream tasks. However, almost all prior studies have only focused on datasets that include single words; thus, how to handle compound words effectively is still a challenge. In this study, we propose a novel method called LERC (Lexical Entailment Recognition Combination) to solve this problem by combining embedding representations and subword semantic features. For this aim, firstly a specialized word embedding model for the LER tasks is trained. Secondly, subword semantic information of word pairs is exploited to compute another feature vector. This feature vector is combined with embedding vectors for supervised classification. We considered three LER tasks, including Lexical Entailment Detection, Lexical Entailment Directionality, and Lexical Entailment Determination. Experimental results conducted on several benchmark datasets in English and Vietnamese languages demonstrated that the subword semantic feature is useful for these tasks. Moreover, LERC outperformed several methods published recently.

Explainable lexical entailment with semantic graphs

AbstractWe present novel methods for detecting lexical entailment in a fully rule-based and explainable fashion, by automatic construction of semantic graphs, in any language for which a crowd-sourced dictionary with sufficient coverage and a dependency parser of sufficient accuracy are available. We experiment and evaluate on both the Semeval-2020 lexical entailment task (Glavaš et al. (2020). Proceedings of the Fourteenth Workshop on Semantic Evaluation, pp. 24–35) and the SherLIiC lexical inference dataset of typed predicates (Schmitt and Schütze (2019). Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 902–914). Combined with top-performing systems, our method achieves improvements over the previous state-of-the-art on both benchmarks. As a standalone system, it offers a fully interpretable model of lexical entailment that makes detailed error analysis possible, uncovering future directions for improving both the semantic parsing method and the inference process on semantic graphs. We release all components of our system as open source software.

Fine-Tuning Word Embeddings for Hierarchical Representation of Data Using a Corpus and a Knowledge Base for Various Machine Learning Applications.

Word embedding models have recently shown some capability to encode hierarchical information that exists in textual data. However, such models do not explicitly encode the hierarchical structure that exists among words. In this work, we propose a method to learn hierarchical word embeddings (HWEs) in a specific order to encode the hierarchical information of a knowledge base (KB) in a vector space. To learn the word embeddings, our proposed method considers not only the hypernym relations that exist between words in a KB but also contextual information in a text corpus. The experimental results on various applications, such as supervised and unsupervised hypernymy detection, graded lexical entailment prediction, hierarchical path prediction, and word reconstruction tasks, show the ability of the proposed method to encode the hierarchy. Moreover, the proposed method outperforms previously proposed methods for learning nonspecialised, hypernym-specific, and hierarchical word embeddings on multiple benchmarks.

Improving Hypernymy Prediction via Taxonomy Enhanced Adversarial Learning

Hypernymy is a basic semantic relation in computational linguistics that expresses the “is-a” relation between a generic concept and its specific instances, serving as the backbone in taxonomies and ontologies. Although several NLP tasks related to hypernymy prediction have been extensively addressed, few methods have fully exploited the large number of hypernymy relations in Web-scale taxonomies.In this paper, we introduce the Taxonomy Enhanced Adversarial Learning (TEAL) for hypernymy prediction. We first propose an unsupervised measure U-TEAL to distinguish hypernymy with other semantic relations. It is implemented based on a word embedding projection network distantly trained over a taxonomy. To address supervised hypernymy detection tasks, the supervised model S-TEAL and its improved version, the adversarial supervised model AS-TEAL, are further presented. Specifically, AS-TEAL employs a coupled adversarial training algorithm to transfer hierarchical knowledge in taxonomies to hypernymy prediction models. We conduct extensive experiments to confirm the effectiveness of TEAL over three standard NLP tasks: unsupervised hypernymy classification, supervised hypernymy detection and graded lexical entailment. We also show that TEAL can be applied to non-English languages and can detect missing hypernymy relations in taxonomies.

Graded hyponymy for compositional distributional semantics

The categorical compositional distributional model of natural language provides a conceptually motivated procedure to compute the meaning of a sentence, given its grammatical structure and the meanings of its words. This approach has outperformed other models in mainstream empirical language processing tasks, but lacks an effective model of lexical entailment. We address this shortcoming by exploiting the freedom in our abstract categorical framework to change our choice of semantic model. This allows us to describe hyponymy as a graded order on meanings, using models of partial information used in quantum computation. Quantum logic embeds in this graded order.

HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment

We introduce HyperLex—a data set and evaluation resource that quantifies the extent of the semantic category membership, that is, type-of relation, also known as hyponymy–hypernymy or lexical entailment (LE) relation between 2,616 concept pairs. Cognitive psychology research has established that typicality and category/class membership are computed in human semantic memory as a gradual rather than binary relation. Nevertheless, most NLP research and existing large-scale inventories of concept category membership (WordNet, DBPedia, etc.) treat category membership and LE as binary. To address this, we asked hundreds of native English speakers to indicate typicality and strength of category membership between a diverse range of concept pairs on a crowdsourcing platform. Our results confirm that category membership and LE are indeed more gradual than binary. We then compare these human judgments with the predictions of automatic systems, which reveals a huge gap between human performance and state-of-the-art LE, distributional and representation learning models, and substantial differences between the models themselves. We discuss a pathway for improving semantic models to overcome this discrepancy, and indicate future application areas for improved graded LE systems.

A Context-Enriched Neural Network Method for Recognizing Lexical Entailment

Recognizing lexical entailment (RLE) always plays an important role in inference of natural language, i.e., identifying whether one word entails another, for example, fox entails animal. In the literature, automatically recognizing lexical entailment for word pairs deeply relies on words' contextual representations. However, as a "prototype" vector, a single representation cannot reveal multifaceted aspects of the words due to their homonymy and polysemy. In this paper, we propose a supervised Context-Enriched Neural Network (CENN) method for recognizing lexical entailment. To be specific, we first utilize multiple embedding vectors from different contexts to represent the input word pairs. Then, through different combination methods and attention mechanism, we integrate different embedding vectors and optimize their weights to predict whether there are entailment relations in word pairs. Moreover, our proposed framework is flexible and open to handle different word contexts and entailment perspectives in the text corpus. Extensive experiments on five datasets show that our approach significantly improves the performance of automatic RLE in comparison with several state-of-the-art methods.

Representing Meaning with a Combination of Logical and Distributional Models

NLP tasks differ in the semantic information they require, and at this time no single semantic representation fulfills all requirements. Logic-based representations characterize sentence structure, but do not capture the graded aspect of meaning. Distributional models give graded similarity ratings for words and phrases, but do not capture sentence structure in the same detail as logic-based approaches. It has therefore been argued that the two are complementary.We adopt a hybrid approach that combines logical and distributional semantics using probabilistic logic, specifically Markov Logic Networks. In this article, we focus on the three components of a practical system:11) Logical representation focuses on representing the input problems in probabilistic logic; 2) knowledge base construction creates weighted inference rules by integrating distributional information with other sources; and 3) probabilistic inference involves solving the resulting MLN inference problems efficiently. To evaluate our approach, we use the task of textual entailment, which can utilize the strengths of both logic-based and distributional representations. In particular we focus on the SICK data set, where we achieve state-of-the-art results. We also release a lexical entailment data set of 10,213 rules extracted from the SICK data set, which is a valuable resource for evaluating lexical entailment systems.2

Dual Density Operators and Natural Language Meaning

Density operators allow for representing ambiguity about a vector representation, both in quantum theory and in distributional natural language meaning. Formally equivalently, they allow for discarding part of the description of a composite system, where we consider the discarded part to be the context. We introduce dual density operators, which allow for two independent notions of context. We demonstrate the use of dual density operators within a grammatical-compositional distributional framework for natural language meaning. We show that dual density operators can be used to simultaneously represent: (i) ambiguity about word meanings (e.g. queen as a person vs. queen as a band), and (ii) lexical entailment (e.g. tiger -> mammal). We provide a proof-of-concept example.

Towards maximal unification of semantically diverse ontologies for controversial domains

Purpose – Ontologies are prone to wide semantic variability due to subjective points of view of their composers. The purpose of this paper is to propose a new approach for maximal unification of diverse ontologies for controversial domains by their relations. Design/methodology/approach – Effective matching or unification of multiple ontologies for a specific domain is crucial for the success of many semantic web applications, such as semantic information retrieval and organization, document tagging, summarization and search. To this end, numerous automatic and semi-automatic techniques were proposed in the past decade that attempt to identify similar entities, mostly classes, in diverse ontologies for similar domains. Apparently, matching individual entities cannot result in full integration of ontologies’ semantics without matching their inter-relations with all other-related classes (and instances). However, semantic matching of ontological relations still constitutes a major research challenge. Therefore, in this paper the authors propose a new paradigm for assessment of maximal possible matching and unification of ontological relations. To this end, several unification rules for ontological relations were devised based on ontological reference rules, and lexical and textual entailment. These rules were semi-automatically implemented to extend a given ontology with semantically matching relations from another ontology for a similar domain. Then, the ontologies were unified through these similar pairs of relations. The authors observe that these rules can be also facilitated to reveal the contradictory relations in different ontologies. Findings – To assess the feasibility of the approach two experiments were conducted with different sets of multiple personal ontologies on controversial domains constructed by trained subjects. The results for about 50 distinct ontology pairs demonstrate a good potential of the methodology for increasing inter-ontology agreement. Furthermore, the authors show that the presented methodology can lead to a complete unification of multiple semantically heterogeneous ontologies. Research limitations/implications – This is a conceptual study that presents a new approach for semantic unification of ontologies by a devised set of rules along with the initial experimental evidence of its feasibility and effectiveness. However, this methodology has to be fully automatically implemented and tested on a larger dataset in future research. Practical implications – This result has implication for semantic search, since a richer ontology, comprised of multiple aspects and viewpoints of the domain of knowledge, enhances discoverability and improves search results. Originality/value – To the best of the knowledge, this is the first study to examine and assess the maximal level of semantic relation-based ontology unification.

Experiments with three approaches to recognizing lexical entailment

AbstractInference in natural language often involves recognizing lexical entailment (RLE), that is, identifying whether one word entails another. For example,buyentailsown. Two general strategies for RLE have been proposed: One strategy is to manually construct an asymmetric similarity measure for context vectors (directional similarity) and another is to treat RLE as a problem of learning to recognize semantic relations using supervised machine-learning techniques (relation classification). In this paper, we experiment with two recent state-of-the-art representatives of the two general strategies. The first approach is an asymmetric similarity measure (an instance of thedirectional similaritystrategy), designed to capture the degree to which the contexts of a word,a, form a subset of the contexts of another word,b. The second approach (an instance of therelation classificationstrategy) represents a word pair,a:b, with a feature vector that is the concatenation of the context vectors ofaandb, and then applies supervised learning to a training set of labeled feature vectors. In addition, we introduce a third approach that is a new instance of therelation classificationstrategy. The third approach represents a word pair,a:b, with a feature vector in which the features are the differences in the similarities ofaandbto a set of reference words. All three approaches use vector space models of semantics, based on word–context matrices. We perform an extensive evaluation of the three approaches using three different datasets. The proposed new approach (similarity differences) performs significantly better than the other two approaches on some datasets and there is no dataset for which it is significantly worse. Along the way, we address some of the concerns raised in past research, regarding the treatment of RLE as a problem of semantic relation classification, and we suggest, it is beneficial to make connections between the research in lexical entailment and the research in semantic relation classification.

Entailment-based Fully Automatic Technique for Evaluation of Summaries

We propose a fully automatic technique for evaluating text summaries without the need to prepare the gold standard summaries man- ually. A standard and popular summary evaluation techniques or tools are not fully automatic; they all need some manual process or manual reference summary. Using recognizing textual entailment (TE), automat- ically generated summaries can be evaluated completely automatically without any manual preparation process. We use a TE system based on a combination of lexical entailment module, lexical distance module, Chunk module, Named Entity module and syntactic text entailment (TE) module. The documents are used as text (T) and summary of these doc- uments are taken as hypothesis (H). Therefore, the more information of the document is entailed by its summary the better the summary. Com- paring with the ROUGE 1.5.5 evaluation scores over TAC 2008 (for- merly DUC, conducted by NIST) dataset, the proposed evaluation tech- nique predicts the ROUGE scores with a accuracy of 98.25% with respect to ROUGE-2 and 95.65% with respect to ROUGE-SU4.

Answering questions about European legislation

ResPubliQA is a Question Answering (QA) evaluation task over European legislation whose first edition was proposed at the Cross Language Evaluation Forum (CLEF) 2009. The exercise consists of extracting a relevant paragraph of text that satisfies the information need expressed by a natural language question. The definition of the task allows to compare current QA technologies with pure Information Retrieval (IR) approaches and to introduce Answer Validation technologies in QA systems. In this paper we describe a system developed for this task. Our system is composed by an IR phase focused on improving QA results, a validation step for removing not promising paragraphs and a module based on n-grams overlapping for selecting the final answer, as well as a selection module that uses Lexical Entailment. While the IR module has contributed to obtain promising results, the performance of the validation module has to be improved. On the other hand, the n-gram ranking improved the results of the ranking given by the IR module.

The Interpretation of Spatial ‘At’: An Experimental Study

This paper presents an experimental study on the interpretation of the spatialpreposition at in adult speakers, based on a variant of the Truth ValueJudgment Task. It is shown that speakers can interpret at as denoting a spatialrelation that stands in the “lexical entailment” relation with other spatial prepositions(e.g. inside, in front of, on top of, behind). For instance, if multiplelocated entities are involved in this relation, then they may occupy locationsthat can be “internal”, “external”, or placed on different verses of the samedirection, e.g. in front or behind a certain landmark object. It is discussed whichsemantic hypothesis correctly predicts these findings, and what the implicationscould be, for a theory of spatial prepositions and their Semantics.

Formalization of linking information in the FrameNet lexicon

The paper presents a novel approach to formalization of linking information in the FrameNet lexicon and to acquisition of a principled syntax-semantics interface, suitable for generalizing over combinatorial properties (valences) of predicators. Focusing on verbs that denote ‘notions’, it adopts an entailment-based view of the concept of semantic role, proposing representations of verbal arguments based on semantically primitive, grammatically relevant properties, entailed by the meaning of predicators (lexical entailments). Such generic meaning components abstract over various semantic relations which humans tend to express systematically through language. A limited set of prototypical role-like concepts can be used for modeling the linking properties of a wide range of verbs, in a well-ordered fashion. In a preliminary study, frame-semantic representations of a set of notion verbs are mapped onto lexical entailment representations, in a portion of the FrameNet corpora. From the annotated data set, associations of semantic and grammatical categories are extracted and are formally rendered in entailment-based classes called Lexicalization Types (L-Types). L-Types are specified in terms of combinations of entailed properties, encoding distinctive predicate-argument structure patterns. A small number of L-Types is shown to readily abstract over the valence patterns of verbs classified in a variety of FrameNet frames. The latter are not systematically connected for purposes of linking. Valence generalizations in the FrameNet lexicon are acquired through appropriate frame-to-frame relations forming the frame hierarchy. L-Types can be represented as abstract, non-lexicalized frames specifying linking constraints. Mappings between L-Types and more specific frames can be encoded by means of a new frame relation modeling the syntax-semantics interface. Such a relation would simplify the current picture of the frame hierarchy by essentially decoupling purely lexical semantic information from information pertaining to linking.

Leveraging Diverse Lexical Resources for Textual Entailment Recognition

Since the problem of textual entailment recognition requires capturing semantic relations between diverse expressions of language, linguistic and world knowledge play an important role. In this article, we explore the effectiveness of different types of currently available resources including synonyms, antonyms, hypernym-hyponym relations, and lexical entailment relations for the task of textual entailment recognition. In order to do so, we develop an entailment relation recognition system which utilizes diverse linguistic analyses and resources to align the linguistic units in a pair of texts and identifies entailment relations based on these alignments. We use the Japanese subset of the NTCIR-9 RITE-1 dataset for evaluation and error analysis, conducting ablation testing and evaluation on hand-crafted alignment gold standard data to evaluate the contribution of individual resources. Error analysis shows that existing knowledge sources are effective for RTE, but that their coverage is limited, especially for domain-specific and other low-frequency expressions. To increase alignment coverage on such expressions, we propose a method of alignment inference that uses syntactic and semantic dependency information to identify likely alignments without relying on external resources. Evaluation adding alignment inference to a system using all available knowledge sources shows improvements in both precision and recall of entailment relation recognition.

A Context-Theoretic Framework for Compositionality in Distributional Semantics

Formalizing “meaning as context” mathematically leads to a new, algebraic theory of meaning, in which composition is bilinear and associative. These properties are shared by other methods that have been proposed in the literature, including the tensor product, vector addition, point-wise multiplication, and matrix multiplication. Entailment can be represented by a vector lattice ordering, inspired by a strengthened form of the distributional hypothesis, and a degree of entailment is defined in the form of a conditional probability. Approaches to the task of recognizing textual entailment, including the use of subsequence matching, lexical entailment probability, and latent Dirichlet allocation, can be described within our framework.