SemEval-2010 Task Research Articles

Modified R-BERT with global semantic information for relation classification task

The objective of the relation classification task is to extract relations between entities. Recent studies have found that R-BERT (Wu and He, 2019) based on pre-trained BERT (Devlin et al., 2019) acquires extremely good results in the relation classification task. However, this method does not take into account the semantic differences between different kinds of entities and global semantic information either. In this paper, we set two different fully connected layers to take into account the semantic difference between subject and object entities. Besides, we build a new module named Concat Module to fully fuse the semantic information among the subject entity vector, object entity vector, and the whole sample sentence representation vector. In addition, we apply the average pooling to acquire a better entity representation of each entity and add the activation operation with a new fully connected layer after our Concat Module. Modifying R-BERT, we propose a new model named BERT with Global Semantic Information (GSR-BERT) for relation classification tasks. We use our approach on two datasets: the SemEval-2010 Task 8 dataset and the Chinese character relationship classification dataset. Our approach achieves a significant improvement over the two datasets. It means that our approach enjoys transferability across different datasets. Furthermore, we prove that these policies we used in our approach also enjoy applicability to named entity recognition task.

Two evaluations on Ontology-style relation annotations

In this paper, we propose an Ontology-Style Relation (OSR) annotation approach. In conventional Relation Extraction (RE) datasets, relations are annotated as a link between two entity mentions. In contrast, in our OSR annotation, a relation is annotated as a relation mention (i.e., not a link but a node) and rdfs:domain and rdfs:range links are annotated from the relation mention to its argument entity mentions. This approach has the following benefits: (1) the relation annotations can be easily converted to Resource Description Framework (RDF) triples to populate an Ontology, (2) some part of conventional RE tasks can be tackled as Named Entity Recognition (NER) tasks, and the relation classes are limited to several RDF properties, and (3) OSR annotations can be used for clear documentations of Ontology contents. We conducted two kinds of evaluation to investigate effects of OSR annotation. We converted (1) an in-house corpus of Japanese Rules of the Road (RoR) in conventional annotations into the OSR annotations and built a novel OSR-RoR corpus and (2) SemEval-2010 Task 8 dataset into the OSR annotations (called OSR-SemEval corpus). We compared the NER and RE performance using neural NER/RE tool DyGIE++ on the conventional and OSR annotations. The experimental results show that the OSR annotations make the RE task easier while introducing slight complexity into the NER task.

Relation Extraction Based on Prompt Information and Feature Reuse

ABSTRACT To alleviate the problem of under-utilization features of sentence-level relation extraction, which leads to insufficient performance of the pre-trained language model and underutilization of the feature vector, a sentence-level relation extraction method based on adding prompt information and feature reuse is proposed. At first, in addition to the pair of nominals and sentence information, a piece of prompt information is added, and the overall feature information consists of sentence information, entity pair information, and prompt information, and then the features are encoded by the pre-trained language model ROBERTA. Moreover, in the pre-trained language model, BIGRU is also introduced in the composition of the neural network to extract information, and the feature information is passed through the neural network to form several sets of feature vectors. After that, these feature vectors are reused in different combinations to form multiple outputs, and the outputs are aggregated using ensemble-learning soft voting to perform relation classification. In addition to this, the sum of cross-entropy, KL divergence, and negative log-likelihood loss is used as the final loss function in this paper. In the comparison experiments, the model based on adding prompt information and feature reuse achieved higher results of the SemEval-2010 task 8 relational dataset.

Improving attention network to realize joint extraction for the construction of equipment knowledge graph

In the context of promoting autonomous decision, intelligent interaction, and concept recognition of the equipment, constructing an equipment Knowledge Graph (KG) is the key technology to facilitate this. In this article, we propose a method that combines design rules and automatic extraction to construct equipment KG from equipment manuals. We define that equipment KG is scheme-based and depicts scheme in the form of KG triples (a.k.a design rules, scheme layer of KG, or concept KG). Our contributions include designing an initial concept KG for the equipment domain and improving the BERT (Bidirectional Encoder Representations from Transformers) model to jointly extract knowledge from texts to enrich the KG. The BERT model was improved from internal calculations so that joint extraction could be achieved directly without extra additional parts. We applied our model to a standard dataset “SemEval2010 Task 8” and achieved the F1 score of 89.55 which demonstrates its rationality. We also established a dataset called “Equipment Manuals Corpus for KG” based on the concept KG and applied the joint model in the dataset to extract knowledge. The result was visualized in the form of a graph.

“Standard Text” Relational Classification Model Based on Concatenated Word Vector Attention and Feature Concatenation

The task of relation classification is an important pre-task in natural language processing tasks. Relation classification can provide a high-quality corpus for tasks such as machine translation, human–computer dialogue, and structured text generation. In the process of the digitalization of standards, identifying the entity relationship in the standard text is an important prerequisite for the formation of subsequent standard knowledge. Only by accurately labeling the relationship between entities can there be higher efficiency and accuracy in the subsequent formation of knowledge bases and knowledge maps. This study proposes a standard text relational classification model based on cascaded word vector attention and feature splicing. The model was compared and ablated on our labeled standard text Chinese dataset. At the same time, in order to prove the performance of the model, the above experiments were carried out on two general English datasets, SemEval-2010 Task 8 and KBP37. On standard text datasets and general datasets, the model proposed in this study achieved excellent results.

Research on Relation Classification Tasks Based on Cybersecurity Text

Relation classification is a significant task within the field of natural language processing. Its objective is to extract and identify relations between two entities in a given text. Within the scope of this paper, we construct an artificial dataset (CS13K) for relation classification in the realm of cybersecurity and propose two models for processing such tasks. For any sentence containing two target entities, we first locate the entities and fine-tune the pre-trained BERT model. Next, we utilize graph attention networks to iteratively update word nodes and relation nodes. A new relation classification model is constructed by concatenating the updated vectors of word nodes and relation nodes. Our proposed model achieved exceptional performance on the SemEval-2010 task 8 dataset, surpassing previous approaches with a remarkable F1 value of 92.3%. Additionally, we propose the integration of a ranking-based voting mechanism into the existing model. Our best results are an F1 value of 92.5% on the SemEval-2010 task 8 dataset and a value 94.6% on the CS13K dataset. These findings highlight the effectiveness of our proposed models in tackling relation classification tasks.

FA-RCNet: A Fused Feature Attention Network for Relationship Classification

Relation extraction is an important task in natural language processing. It plays an integral role in intelligent question-and-answer systems, semantic search, and knowledge graph work. For this task, previous studies have demonstrated the effectiveness of convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long short-term memory networks (LSTMs) in relational classification tasks. Recently, due to the superior performance of the pre-trained model BERT, BERT has become a feature extraction module for many relational classification models, and good results have been achieved in work related to BERT. However, most of such work uses the deepest levels of features. The important role of shallow-level information in the relational classification task is ignored. Based on the above problems, a relationship classification network FA-RCNet (fusion-attention relationship classification network) with feature fusion and attention mechanism is proposed in this paper. FA-RCNet fuses shallow-level features with deep-level features, and augments entity features and global features by the attention module so that the feature vector can perform the relational classification task more perfectly. In addition, the model in this paper achieves advanced results on both the SemEval-2010 Task 8 dataset and the KBP37 dataset compared to previously published models.

CRSAtt: By Capturing Relational Span and Using Attention for Relation Classification

Relation classification is an important fundamental task in information extraction, and convolutional neural networks have been commonly applied to relation classification with good results. In recent years, due to the proposed pre-training model BERT, the use of which as a feature extraction architecture has become more and more popular, convolutional neural networks have gradually withdrawn from the stage of NLP, and the relation classification/extraction model based on pre-training BERT has achieved state-of-the-art results. However, none of these methods consider how to accurately capture the semantic features of the relationships between entities to reduce the number of noisy words in a sentence that are not helpful for relation classification. Moreover, these methods do not have a systematic prediction structure to fully utilize the extracted features for the relational classification task. To address these problems, a SpanBert-based relation classification model is proposed in this paper. Compared with existing Bert-based architectures, the model is able to understand the semantic information of the relationships between entities more accurately, and it can fully utilize the extracted features to represent the degree of dependency of a pair of entities with each type of relationship. In this paper, we design a feature fusion method called “SRS” (Strengthen Relational Semantics) and an attention-based prediction structure. Compared with existing methods, the feature fusion method proposed in this paper can reduce the noise interference of irrelevant words when extracting relational semantics, and the prediction structure proposed in this paper can make full use of semantic features for relational classification. We achieved advanced results on the SemEval-2010 Task 8 and the KBP37 relational dataset.

A Two-channel model for relation extraction using multiple trained word embeddings

As an essential task in the field of knowledge graph, relation extraction (RE) has received extensive attention from researchers. Since the existing RE methods only adopt one trained word embedding to obtain sentence representation, the polysemy problem cannot be well solved. In order to alleviate the polysemy in RE, this paper proposes a Two-channel model by adopting multiple trained word embeddings, in which one channel is a bidirectional long-short-term memory network based on an attention mechanism (Bi-LSTM-ATT), and the other channel is a convolutional neural network (CNN). Furthermore, a two-channel fusion method is proposed based on this model to deal with polysemy problem in RE. As a result, the Two-channel model achieves 85.42% and 62.2% F1-scores on the Semeval-2010 Task 8 dataset and KBP37 dataset, respectively. The experiment results show that the Two-channel model performs better than most existing models under the condition without using the external features generated by natural language processing (NLP) tools. On the other hand, the two-channel fusion method also obtains a better performance than either concatenation or addition on the two channels.

Relationship classification based on dependency parsing and the pretraining model

As an important part of information extraction, relationship extraction aims to extract the relationships between given entities from natural language text. Based on the pretraining model R-BERT, this paper proposes an entity relationship extraction method that integrates an entity dependency path and pretraining model, which generates a dependency parse tree by dependency parsing, obtains the dependency path of an entity pair via a given entity, and uses an entity dependency path to exclude information such as modifier chunks and useless entities in sentences. This model has achieved good F1 value performance on the SemEval2010 Task 8 dataset. Experiments on datasets show that dependency parsing can provide context information for models and improve performance.

Adversarial training for supervised relation extraction

Most supervised methods for relation extraction (RE) involve time-consuming human annotation. Distant supervision for RE is an efficient method to obtain large corpora that contains thousands of instances and various relations. However, the existing approaches rely heavily on knowledge bases (e.g., Freebase), thereby introducing data noise. Various relations and noisy labeling instances make the issue difficult to solve. In this study, we propose a model based on a piecewise convolution neural network with adversarial training. Inspired by generative adversarial networks, we adopt a heuristic algorithm to identify noisy datasets and apply adversarial training to RE. Experiments on the extended dataset of SemEval-2010 Task 8 show that our model can obtain more accurate training data for RE and significantly outperforms several competitive baseline models. Our model has an Fi score of 89.61%.

Contextual Semantic-Guided Entity-Centric GCN for Relation Extraction

Relation extraction tasks aim to predict potential relations between entities in a target sentence. As entity mentions have ambiguity in sentences, some important contextual information can guide the semantic representation of entity mentions to improve the accuracy of relation extraction. However, most existing relation extraction models ignore the semantic guidance of contextual information to entity mentions and treat entity mentions in and the textual context of a sentence equally. This results in low-accuracy relation extractions. To address this problem, we propose a contextual semantic-guided entity-centric graph convolutional network (CEGCN) model that enables entity mentions to obtain semantic-guided contextual information for more accurate relational representations. This model develops a self-attention enhanced neural network to concentrate on the importance and relevance of different words to obtain semantic-guided contextual information. Then, we employ a dependency tree with entities as global nodes and add virtual edges to construct an entity-centric logical adjacency matrix (ELAM). This matrix can enable entities to aggregate the semantic-guided contextual information with a one-layer GCN calculation. The experimental results on the TACRED and SemEval-2010 Task 8 datasets show that our model can efficiently use semantic-guided contextual information to enrich semantic entity representations and outperform previous models.

Novel target attention convolutional neural network for relation classification

Relation classification (RC) is an essential task in natural language processing (NLP), which extracts relationships of entity pairs in sentences of text. In the paper, a novel target attention convolutional neural network (TACNN) is proposed for the RC by fully utilizing word embedding information and position embedding information. Simultaneously, a target attention mechanism (TAM) is applied into a context layer of the convolutional neural network (CNN) model, which increases the effect of the relationship matrix weights of two entities in the sentence, while ignoring the calculation of irrelevant terms. And the TACNN is essentially to modify the weight of the relationship matrix of entities in the sentence at the context layer and connect the relationship feature composed of the lexical layer feature with the target attention layer feature. Therefore, the TACNN simplifies the structure of the CNN and improves the computational efficiency. On SemEval-2010 Task 8 dataset and Conll04 dataset, the TACNN obtains 85.3% and 71.4% of the F1-score, respectively. In contrast to previously available public models, the TACNN achieves a state-of-the-art level in the F1-score of the RC.

BERT gated multi-window attention network for relation extraction

Entity relation extraction aims to identify the semantic relation between entity pairs in a sentence and is an important technical support for downstream tasks such as question answering systems and semantic searching. Existing relation extraction models mainly rely on neural networks to extract the semantic information of sentences, ignoring the critical role of important phrase information in relation extraction. For this problem, this paper proposes a relation extraction model based on BERT gated multi-window attention network (BERT-GMAN). The model first uses BERT to extract the semantic representation features of the sentence and its constraint information. Secondly, it constructs the key phrases extraction network to obtain multi-granularity phrase information and uses element-wise max pooling to select key phrases features. Thirdly, it adopts classification feature perception network to further filter and globally perceive key phrase feature to form the overall features of relation classification. Finally, it combines with Softmax classifier to perform relation extraction. The experimental results on the Semeval-2010 Task 8 dataset show that the performance of the model in this paper is further improved compared with the existing methods, and the F1-score reaches 90.25%.

Relation classification via BERT with piecewise convolution and focal loss.

Recent relation extraction models’ architecture are evolved from the shallow neural networks to natural language model, such as convolutional neural networks or recurrent neural networks to Bert. However, these methods did not consider the semantic information in the sequence or the distance dependence problem, the internal semantic information may contain the useful knowledge which can help relation classification. Focus on these problems, this paper proposed a BERT-based relation classification method. Compare with the existing Bert-based architecture, the proposed model can obtain the internal semantic information between entity pair and solve the distance semantic dependence better. The pre-trained BERT model after fine tuning is used in this paper to abstract the semantic representation of sequence, then adopt the piecewise convolution to obtain semantic information which influence the extraction results. Compare with the existing methods, the proposed method can achieve a better accuracy on relational extraction task because of the internal semantic information extracted in the sequence. While, the generalization ability is still a problem that cannot be ignored, and the numbers of the relationships are difference between different categories. In this paper, the focal loss function is adopted to solve this problem by assigning a heavy weight to less number or hard classify categories. Finally, comparing with the existing methods, the F1 metric of the proposed method can reach a superior result 89.95% on the SemEval-2010 Task 8 dataset.

Cross-lingual transfer learning for relation extraction using Universal Dependencies

This paper focuses on the task of cross-language relation extraction, which aims to identify the semantic relations holding between entities in the text. The goal of the task is to train classifiers for low-resource languages by means of the annotated data from high-resource languages. Related methods usually employ parallel data or Machine Translator (MT) to project annotated data from a source to a target language. However, the availability and the quality of parallel data and MT are big challenges for low-resource languages. In this paper, a novel transfer learning method is presented for this task. The key idea is to utilize a tree-based representation of data, which is highly informative for classifying semantic relations, and also shared among different languages. All the training and test data are shown using this representation. We propose to use the Universal Dependency (UD) parsing, which is a language-agnostic formalism for representation of syntactic structures. Equipping UD parse trees with multi-lingual word embeddings makes an ideal representation for the cross-language relation extraction task. We propose two deep networks to use this representation. The first one utilizes the Shortest Dependency Path of UD trees, while the second employs the UD-based positional embeddings. Experiments are performed using SemEval 2010-task 8 training data, whereas French and Farsi are the test languages. The results show 63.9% and 56.2% F1 scores, for French and Farsi test data, respectively, which are 14.4% and 17.9% higher than the baseline. This work can be considered a simple yet powerful baseline for further investigation into the cross-language tasks.

Entity-Centric Fully Connected GCN for Relation Classification

Relation classification is an important task in the field of natural language processing, and it is one of the important steps in constructing a knowledge graph, which can greatly reduce the cost of constructing a knowledge graph. The Graph Convolutional Network (GCN) is an effective model for accurate relation classification, which models the dependency tree of textual instances to extract the semantic features of relation mentions. Previous GCN based methods treat each node equally. However, the contribution of different words to express a certain relation is different, especially the entity mentions in the sentence. In this paper, a novel GCN based relation classifier is propose, which treats the entity nodes as two global nodes in the dependency tree. These two global nodes directly connect with other nodes, which can aggregate information from the whole tree with only one convolutional layer. In this way, the method can not only simplify the complexity of the model, but also generate expressive relation representation. Experimental results on two widely used data sets, SemEval-2010 Task 8 and TACRED, show that our model outperforms all the compared baselines in this paper, which illustrates that the model can effectively utilize the dependencies between nodes and improve the performance of relation classification.

Selection gate-based networks for semantic relation extraction

Semantic relatedness between context information and entities, which is one of the most easily accessible features, has been proven to be very useful for detecting the semantic relation held in the text segment. However, some methods fail to take into account important information between entities and contexts. How to effectively choose the closest and the most relevant information to the entity in context words in a sentence is an important task. In this paper, we propose selection gate-based networks (SGate-NN) to model the relatedness of an entity word with its context words, and select the relevant parts of contexts to infer the semantic relation toward the entity. We conduct experiments using the SemEval-2010 Task 8 dataset. Extensive experiments and the results demonstrate that the proposed method is effective for relation classification, which can obtain state-of-the-art classification accuracy.

Auxiliary Learning for Relation Extraction

Relation extraction aims to predict a semantic relation between entities in a sentence, which is usually regarded as a classification problem. However, due to the limited relation set, many semantic relations are labeled as a special artificial relation type, termed <inline-formula><tex-math notation="LaTeX">$no\_relation$</tex-math></inline-formula>, if they are beyond the predefined relation set. Existing methods treat this artificial relation type <inline-formula><tex-math notation="LaTeX">$no\_relation$</tex-math></inline-formula> as a common semantic relation without taking its rich semantics into account. In this paper, a novel auxiliary learning method is proposed to excavate the semantic information of <inline-formula><tex-math notation="LaTeX">$no\_relation$</tex-math></inline-formula>, resulting in the improvement of generalization performance. The auxiliary learning method focuses on the model learning phase and introduces a binary classification task that treats the artificial relation type <inline-formula><tex-math notation="LaTeX">$no\_relation$</tex-math></inline-formula> as negative class and the rest semantic types as positive class. The binary classification task, named auxiliary learning task, pays more attention to <inline-formula><tex-math notation="LaTeX">$no\_relation$</tex-math></inline-formula> with a cost-sensitive loss by assigning higher cost on the misclassification of negative samples than positive ones. An additional reward is provided to the main prediction task by the auxiliary learning task, which leads to a better representation for relation extraction. Significant improvements are consistently achieved when state-of-the-art models are equipped with the auxiliary learning task on SemEval-2010 Task 8 and the large-scale TACRED. Especially, new state-of-the-art performance is achieved on SemEval-2010 Task 8 by the proposed method. Meanwhile, the motivation for introducing the auxiliary learning method is further reinforced by extensive experiments.

Relation Extraction using Language Model Based on Knowledge Graph

Relation extraction is an important task in natural language processing (NLP). The existing methods generally pay more attention on extracting textual semantic information from text, but ignore the relation contextual information from existed relations in datasets, which is very important for the performance of relation extraction task. In this paper, we represent each individual entity as a embedding based on entities and relations knowledge graph, which encodes the relation contextual information between the given entity pairs and relations. Besides, inspired by the impressive performance of language models recently, we used the language model to leverage word semantic information, in which word semantic information can be better captured than word embedding. The experimental results on SemEval2010 Task 8 dataset showed that the F1-score of our proposed method improved nearly 3% compared with the previous methods.