Semantic Relation Classification Research Articles

Fine-tuning language models to recognize semantic relations

Transformer-based pre-trained Language Models (PLMs) have emerged as the foundations for the current state-of-the-art algorithms in most natural language processing tasks, in particular when applied to context rich data such as sentences or paragraphs. However, their impact on the tasks defined in terms of abstract individual word properties, not necessary tied to their specific use in a particular sentence, has been inadequately explored, which is a notable research gap. Addressing this gap is crucial for advancing our understanding of natural language processing. To fill this void, we concentrate on classification of semantic relations: given a pair of concepts (words or word sequences) the aim is to identify the semantic label to describe their relationship. E.g. in the case of the pair green/colour, “is a” is a suitable relation while “part of”, “property of”, and “opposite of” are not suitable. This classification is independent of a particular sentence in which these concepts might have been used. We are first to incorporate a language model into both existing approaches to this task, namely path-based and distribution-based methods. Our transformer-based approaches exhibit significant improvements over the state-of-the-art and come remarkably close to achieving human-level performance on rigorous benchmarks. We are also first to provide evidence that the standard datasets over-state the performance due to the effect of “lexical memorisation.” We reduce this effect by applying lexical separation. On the new benchmark datasets, the algorithmic performance remains significantly below human-level, highlighting that the task of semantic relation classification is still unresolved, particularly for language models of the sizes commonly used at the time of our study. We also identify additional challenges that PLM-based approaches face and conduct extensive ablation studies and other experiments to investigate the sensitivity of our findings to specific modelling and implementation choices. Furthermore, we examine the specific relations that pose greater challenges and discuss the trade-offs between accuracy and processing time.

BertSRC: transformer-based semantic relation classification

The relationship between biomedical entities is complex, and many of them have not yet been identified. For many biomedical research areas including drug discovery, it is of paramount importance to identify the relationships that have already been established through a comprehensive literature survey. However, manually searching through literature is difficult as the amount of biomedical publications continues to increase. Therefore, the relation classification task, which automatically mines meaningful relations from the literature, is spotlighted in the field of biomedical text mining. By applying relation classification techniques to the accumulated biomedical literature, existing semantic relations between biomedical entities that can help to infer previously unknown relationships are efficiently grasped. To develop semantic relation classification models, which is a type of supervised machine learning, it is essential to construct a training dataset that is manually annotated by biomedical experts with semantic relations among biomedical entities. Any advanced model must be trained on a dataset with reliable quality and meaningful scale to be deployed in the real world and can assist biologists in their research. In addition, as the number of such public datasets increases, the performance of machine learning algorithms can be accurately revealed and compared by using those datasets as a benchmark for model development and improvement. In this paper, we aim to build such a dataset. Along with that, to validate the usability of the dataset as training data for relation classification models and to improve the performance of the relation extraction task, we built a relation classification model based on Bidirectional Encoder Representations from Transformers (BERT) trained on our dataset, applying our newly proposed fine-tuning methodology. In experiments comparing performance among several models based on different deep learning algorithms, our model with the proposed fine-tuning methodology showed the best performance. The experimental results show that the constructed training dataset is an important information resource for the development and evaluation of semantic relation extraction models. Furthermore, relation extraction performance can be improved by integrating our proposed fine-tuning methodology. Therefore, this can lead to the promotion of future text mining research in the biomedical field.

A deep learning based method benefiting from characteristics of patents for semantic relation classification

• The unique characteristics of patents are highlighted via comparison. • A customized deep-learning model is proposed to leverage such characteristics. • The connection between entity pairs is measured on the basis of associated rules. • The proposed model outperforms prior models on semantic relation classification. The deep learning has become an important technique for semantic relation classification in patent texts. Previous studies just borrowed the relevant models from generic texts to patent texts while keeping structure of the models unchanged. Due to significant distinctions between patent texts and generic ones, this enables the performance of these models in the patent texts to be reduced dramatically. To highlight these distinct characteristics in patent texts, seven annotated corpora from different fields are comprehensively compared in terms of several indicators for linguistic characteristics. Then, a deep learning based method is proposed to benefit from these characteristics. Our method exploits the information from other similar entity pairs as well as that from the sentences mentioning a focal entity pair. The latter stems from the conventional practices, and the former from our meaningful observation: the stronger the connection between two entity pairs is, the more likely they belong to the same relation type. To measure quantitatively the connection between two entity pairs, a similarity indicator on the basis of association rules is raised. Extensive experiments on the corpora of TFH-2020 and ChemProt demonstrate that our method for semantic relation classification is capable of benefiting from characteristic of patent texts.

Characterization and classification of semantic image-text relations

The beneficial, complementary nature of visual and textual information to convey information is widely known, for example, in entertainment, news, advertisements, science, or education. While the complex interplay of image and text to form semantic meaning has been thoroughly studied in linguistics and communication sciences for several decades, computer vision and multimedia research remained on the surface of the problem more or less. An exception is previous work that introduced the two metrics Cross-Modal Mutual Information and Semantic Correlation in order to model complex image-text relations. In this paper, we motivate the necessity of an additional metric called Status in order to cover complex image-text relations more completely. This set of metrics enables us to derive a novel categorization of eight semantic image-text classes based on three dimensions. In addition, we demonstrate how to automatically gather and augment a dataset for these classes from the Web. Further, we present a deep learning system to automatically predict either of the three metrics, as well as a system to directly predict the eight image-text classes. Experimental results show the feasibility of the approach, whereby the predict-all approach outperforms the cascaded approach of the metric classifiers.

Semantic Relation Classification via Bidirectional LSTM Networks with Entity-Aware Attention Using Latent Entity Typing

Classifying semantic relations between entity pairs in sentences is an important task in natural language processing (NLP). Most previous models applied to relation classification rely on high-level lexical and syntactic features obtained by NLP tools such as WordNet, the dependency parser, part-of-speech (POS) tagger, and named entity recognizers (NER). In addition, state-of-the-art neural models based on attention mechanisms do not fully utilize information related to the entity, which may be the most crucial feature for relation classification. To address these issues, we propose a novel end-to-end recurrent neural model that incorporates an entity-aware attention mechanism with a latent entity typing (LET) method. Our model not only effectively utilizes entities and their latent types as features, but also builds word representations by applying self-attention based on symmetrical similarity of a sentence itself. Moreover, the model is interpretable by visualizing applied attention mechanisms. Experimental results obtained with the SemEval-2010 Task 8 dataset, which is one of the most popular relation classification tasks, demonstrate that our model outperforms existing state-of-the-art models without any high-level features.

ANALYSES OF THE DEPENDENT CLAUSE IN ASYNDETIC COMPOUND SENTENCES

The given article presents the views on the dependent clause in asyndetic compound sentences and their broad variants of usage. The semantic relations between the components of the asyndetic compound sentences are shown, where the second component directly defines the meaning of the first component of the sentences. The classification of semantic relations between the two components of asyndetic compound sentences with dependent clause in the Uzbek language is presented by examples. Furthermore, the syntactic functions of the subordinate clause in the asyndetic compound sentences are also the matter of investigations. Keywords: compound sentences, dependent clause, dependent clause in asyndetic compound sentence, semantics of the compound sentences, grammatical forms of the asyndetic compound sentences.

Semantic relation classification through low-dimensional distributed representations of partial word sequences

Semantic relation classification through low-dimensional distributed representations of partial word sequences

Classification of composite semantic relations by a distributional-relational model

Different semantic interpretation tasks such as text entailment and question answering require the classification of semantic relations between terms or entities within text. However, in most cases, it is not possible to assign a direct semantic relation between entities/terms. This paper proposes an approach for composite semantic relation classification using one or more relations between entities/term mentions, extending the traditional semantic relation classification task. The proposed model is different from existing approaches which typically use machine learning models built over lexical and distributional word vector features in that is uses a combination of a large commonsense knowledge base of binary relations, a distributional navigational algorithm and sequence classification to provide a solution for the composite semantic relation classification problem. The proposed approach outperformed existing baselines with regard to F1-score, Accuracy, Precision and Recall.

Semantic Hashtag Relation Classification Using Co-occurrence Word Information

Users using social networking service (SNS) may express their thoughts and feelings using simple hashtags. Hashtags are related to other hashtags and images that are used together in the user’s other posts. Understanding the meaning of personal hashtags can be a way to learn latent semantic expressions of personal words. Existing methods for learning and analyzing semantics such as Latent Semantic Analysis, Latent Dirichlet Allocation and Word Embedding need large-scale corpus to construct an elaborate model. Large-scale corpus usually consists of words that a lot of people already use. Thus, existing methods are able to catch the latent meaning of words used in general. However, it is difficult for these methods to find personal meanings of words that are used by a particular person. Because the number of words that a person use is usually very small compared to a large-scale corpus. Another reason for the difficulty is that existing methods use occurrence frequency or co-occurrence probability. Therefore, the importance or the frequency or the probability of personalized meaning may disappear because of this large difference in the number of words. In this research we focused on the classification of semantic words using a user’s hashtag data and the co-occurrence of these hashtags. The performance is evaluated and enhances previous work by 18% for Precision and more than 70% for Recall.

Классификация семантических отношений компонентов двухступенчатого эпитета в парадигматическом и синтагматическом аспектах

Классификация семантических отношений компонентов двухступенчатого эпитета в парадигматическом и синтагматическом аспектах

Мовносоціумний простір прийменника: дискурсивні практики.

Modern Ukrainian language is characterized by interrelated tendencies of synthetical character and analyticity, which are motivated by: 1) the folk-colloquial element of the literary norm; 2) book tradition; 3) the law of language economy, etc. One of the brightest analysts expresses the dynamic development of the prepositional system, which has recently been actively replenished by semantically specialized two-component, three-component, and other entities. The ultimate manifestation of such an analyticism is the presence of dissected prepositions of the sample від... до, з ... до.The textual function of the prepositions within the texts appears in the inter-phrased / intra-phrased, intersentenced manifestations; at the same time, the expansion of the paradigmatic plane of prepositions, their participation in the creation of images, are apparent. Within the context the preposition appears as a relatively independent element with its own inventory of distributions, phonetic variants, and others. The solution of the question of the status of the text of the prepositions will enable understanding of the mechanisms of interaction between the components and components of the language system and the establishment of ways for the creation and functioning of syntaxemes, the extension of the formation of their own semantic potential of the latter.Lexical and grammatical meanings of prepositions coincide, which does not mean their identity. The grammatical meaning of the preposition is the realization of the form of syntactic relation between the words, and the lexical one should consider the designation of a certain relation between the objects, the action and the object, etc., which makes it possible to enumerate the prepositions to the words-relatives. The ability of prepositions to determine its lexical meaning in the noun (more broadly – in the name) is related not to the absence of this value, but to its corresponding specifics. Interpretation of prepositions should be based on the lexical environment, since they indicate the relation between the objects. The ratio implies the presence of not less than two quantities, therefore, prepositions can not exist without these values, that is, they can not be used independently. The meaning of the prepositions implies their functioning within the phrase. And by its nature the primary prepositions are many-valued, they are characterized by homonymy. In this case, the context is a diagnostic indicator of a certain value as a virtual-system.In analyzing the semantics of relations, it is necessary to take into account, as much as possible, the particularities of the lexical meaning of prepositions and the semantic features of words that form the left and right-side distribution of the prepositional-case design. Due to this, the following classification of intratextual semantic relations, expressed by the Ukrainian primary prepositions in artistic-fiction and journalistic language-linguistic discourse practices, is real: 1) spatial relationships that are the most researched in modern linguistics. Among them differentiate the local (place) and additive (direction). Locative relations are differentiated into: suppressive (finding above the surface) and invasive (finding inside).

Возможности использования сервиса RusVectōrēs для выявления семантических ассоциатов глаголов русского языка

The paper presents the results of investigating distributional semantics’ potential in its realization in the form Web-service RusVectōrēs. The question about applying the service for studying semantics of Russian verbs is considered. The research urgency is caused by insufficient level of information about linguistic possibilities of distributional semantics in whole. The novelty of the research consist in the fact that the question about applying the instrument RusVectōrēs for investigating verb semantics is posed for the first time. Classification of semantic relations extracted by RusVectōrēs for Russian verbs is proposed. The analyzed data include two lists of entry verbs and the set of semantic associates for each verb; the integral set of considered semantic associates consists of 468 verbs. Special attention is paid to semantic relations that can be treated as lexical functions because this sort of relations appeared to be the most frequent for Russian verbs; the whole number of extracted lexical functions is equal to 28. It is shown that lexical functions that correspond to aspectual variants of verbs, to synonymic relations and conversion are the most frequent; hyponyms and co-hyponyms are the most frequent among semantic relations that differ from lexical functions; situational relations and actant relations are comparatively rare. Special attention is paid to the details that are important for applying the instruments of lexical functions to semantic relations extracted for verbs by the service RusVectōrēs.

Adaptive Multi-Compositionality for Recursive Neural Network Models

Recursive neural network models have achieved promising results in many natural language processing tasks. The main difference among these models lies in the composition function, i.e., how to obtain the vector representation for a phrase or sentence using the representations of words it contains. This paper introduces a novel Adaptive Multi-Compositionality (AdaMC) layer to recursive neural network models. The basic idea is to use more than one composition function and adaptively select them depending on input vectors. We develop a general framework to model the semantic composition as a distribution of these composition functions. The composition functions and parameters used for adaptive selection are jointly learnt from the supervision of specific tasks. We integrate AdaMC into existing recursive neural network models and conduct extensive experiments on the Stanford Sentiment Treebank and semantic relation classification task. The experimental results demonstrate that AdaMC improves the performance of recursive neural network models and outperforms the baseline methods.

An empirical convolutional neural network approach for semantic relation classification

In industry, relation classification plays a significant role in today׳s search engine. Up to now, the state-of-the-art systems have the problems of over-reliance on the quality of handcrafted features annotated by experts and linguistic knowledge derived from linguistic analysis modules, which is costly and leads to the issue of error propagation. Currently, with the data-driven approaches attracting wide attention, deep learning achieves impressive performance in semantic processing tasks without much effort on costly features. In this work, we deal with the relation classification task utilizing a convolutional neural network (CNN) approach to automatically control feature learning from raw sentences and minimize the application of external toolkits and resources. Our proposed method has several distinct features. First, we exploit a simple but rational way to specify which input tokens are the target nominals in the input sentence, instead of Position Feature that used in other neural network relation classification systems. Secondly, a most suitable dropout strategy is used to prevent units in the neural network from co-adapting too much, which significantly reduces over-fitting and improves the performance. Eventually, using only word embedding as input features is sufficient to achieve desirable performance. Our experiments on the SemEval-2010 Task-8 dataset show that our CNN architecture without using any additional extracted features significantly outperforms the state-of-the-art systems and achieves an F1-score of 84.8% only considering the context between the two target nominals.

Automatic extraction and visualization of semantic relations between medical entities from medicine instructions

Recent years have witnessed the rapid development and tremendous research interests in healthcare domain. The health and medical knowledge can be acquired from many sources, such as professional health providers, health community generated data and textual descriptions of medicines. This paper explores the classification and extraction of semantic relation between medical entities from the unstructured medicine Chinese instructions. In this paper, three kinds of textual features are extracted from medicine instruction according to the nature of natural language texts. And then, a support vector machine based classification model is proposed to categorize the semantic relations between medical entities into the corresponding semantic relation types. Finally, the extraction algorithm is utilized to obtain the semantic relation triples. This paper also visualizes the semantic relations between medical entities with relationship graph for their future processing. The experimental results show that the approach proposed in this paper is effective and efficient in the classification and extraction of semantic relations between medical entities.

Experiments with three approaches to recognizing lexical entailment

AbstractInference in natural language often involves recognizing lexical entailment (RLE), that is, identifying whether one word entails another. For example,buyentailsown. Two general strategies for RLE have been proposed: One strategy is to manually construct an asymmetric similarity measure for context vectors (directional similarity) and another is to treat RLE as a problem of learning to recognize semantic relations using supervised machine-learning techniques (relation classification). In this paper, we experiment with two recent state-of-the-art representatives of the two general strategies. The first approach is an asymmetric similarity measure (an instance of thedirectional similaritystrategy), designed to capture the degree to which the contexts of a word,a, form a subset of the contexts of another word,b. The second approach (an instance of therelation classificationstrategy) represents a word pair,a:b, with a feature vector that is the concatenation of the context vectors ofaandb, and then applies supervised learning to a training set of labeled feature vectors. In addition, we introduce a third approach that is a new instance of therelation classificationstrategy. The third approach represents a word pair,a:b, with a feature vector in which the features are the differences in the similarities ofaandbto a set of reference words. All three approaches use vector space models of semantics, based on word–context matrices. We perform an extensive evaluation of the three approaches using three different datasets. The proposed new approach (similarity differences) performs significantly better than the other two approaches on some datasets and there is no dataset for which it is significantly worse. Along the way, we address some of the concerns raised in past research, regarding the treatment of RLE as a problem of semantic relation classification, and we suggest, it is beneficial to make connections between the research in lexical entailment and the research in semantic relation classification.

General and specific paraphrases of semantic relations between nouns

AbstractMany English noun pairs suggest an almost limitless array of semantic interpretation. A fruit bowl might be described as a bowl for fruit, a bowl that contains fruit, a bowl for holding fruit, or even (perhaps in a modern sculpture class), a bowl made out of fruit. These interpretations vary in syntax, semantic denotation, plausibility, and level of semantic detail. For example, a headache pill is usually a pill for preventing headaches, but might, perhaps in the context of a list of side effects, be a pill that can cause headaches (Levi, J. N. 1978. The Syntax and Semantics of Complex Nominals. New York: Academic Press.). In addition to lexical ambiguity, both relational ambiguity and relational vagueness make automatic semantic interpretation of these combinations difficult. While humans parse these possibilities with ease, computational systems are only recently gaining the ability to deal with the complexity of lexical expressions of semantic relations. In this paper, we describe techniques for paraphrasing the semantic relations that can hold between nouns in a noun compound, using a semi-supervised probabilistic method to rank candidate paraphrases of semantic relations, and describing a new method for selecting plausible relational paraphrases at arbitrary levels of semantic specification. These methods are motivated by the observation that existing semantic relation classification schemes often exhibit a highly skewed class distribution, and that lexical paraphrases of semantic relations vary widely in semantic precision.

SET EXPANSION OF CONTEXTUAL SEMANTIC RELATIONS: AN ALTERNATIVE TO FULL CORPUS ANNOTATION FOR SUPERVISED CLASSIFICATION

Recent approaches for classification of semantic relations are based on supervised learning using large training datasets. Due to the high cost of annotating such data and to the class imbalance problem, alternatives for minimizing the effort of full corpus annotation are required. In set expansion, one of such alternatives, given a small initial training set, new relevant instances are acquired from a large corpus. However, when dealing with contextual semantic relations, which are relations that are highly dependent on the context within the sentence, set expansion is not trivial, since instances are not directly queryable and filtering requires classification under a very restricted number of training instances. This work thus proposes a bootstrapped set expansion method for contextual semantic relations. It performs a best effort extraction using the Web, and a two-stage filtering of candidate instances, the first based on syntactic patterns and the second using a feature distance-based classifier designed for the low frequency setting. The relevance of the output is measured experimentally by using the expanded set as the training data of the supervised classification task, observing an incremental improvement in performance after each bootstrapping iteration when compared to values using the unexpanded training data.

Kernel-based semantic relation detection and classification via enriched parse tree structure

This paper proposes a tree kernel method of semantic relation detection and classification (RDC) between named entities. It resolves two critical problems in previous tree kernel methods of RDC. First, a new tree kernel is presented to better capture the inherent structural information in a parse tree by enabling the standard convolution tree kernel with context-sensitiveness and approximate matching of sub-trees. Second, an enriched parse tree structure is proposed to well derive necessary structural information, e.g., proper latent annotations, from a parse tree. Evaluation on the ACE RDC corpora shows that both the new tree kernel and the enriched parse tree structure contribute significantly to RDC and our tree kernel method much outperforms the state-of-the-art ones.

Kernel-based semantic relation detection and classification via enriched parse tree structure

Kernel-based semantic relation detection and classification via enriched parse tree structure