Entity Relation Extraction Research Articles

Relation Extraction Based on BERT and BGRU in the Chinese Music Scene

Due to the lack of music knowledge extraction tools and public datasets, there are relatively few studies on music entity relation extraction. The Bidirectional Encoder Representations from Transformer (BERT) model considers contextual information and provides more dynamic, richer vector representations of generated words. Bidirectional Gated Recurrent Units (BGRU) has longdistance learning capability, which can obtain rich contextual semantic feature. Therefore, this paper proposes a deep learning model based on BERT and BGRU combined with an attention mechanism for Chinese music relation extraction. Experimental results show that the proposed method obtains good results when extracting the relation between musical entities.

The Entity Relationship Extraction Method Using Improved RoBERTa and Multi-Task Learning

The Entity Relationship Extraction Method Using Improved RoBERTa and Multi-Task Learning

Retracted: An Entity Relationship Extraction Model Based on BERT-BLSTM-CRF for Food Safety Domain.

[This retracts the article DOI: 10.1155/2022/7773259.].

Joint Extraction of Organizations and Relations for Emergency Response Plans With Rich Semantic Information Based On Multi-Head Attention Mechanism

At present, deep learning-based joint entity-relation extraction models are gradually able to accomplish complex tasks, but the research progress in specific fields is relatively slow. Compared with other fields, emergency plan text has the characteristics of high entity density, long text, and many professional terms, which make some general models unable to handle the semantic information of emergency plan text well. Therefore, this paper addresses the problem of complex semantics of emergency plan text, and proposes a joint extraction model of emergency plan organization and relationship based on multi-Head Attention Mechanism (MA-JE) to enrich semantic information, starting from multiple perspectives and different levels to obtain contextual information, aiming to deeply mine and use sentence semantic information through deep feature extraction of emergency plan text. The proposed model and the baseline model are experimented separately on the Chinese emergency response plan dataset, and the results show that the proposed approach outperforms existing baseline models for joint extraction of entity and their relations. In addition, ablation experiments were performed to verify the validity of each module in the model.

A hybrid attention mechanism for multi-target entity relation extraction using graph neural networks

Relation extraction is a cardinal natural language processing task employed to process a given corpus and infer hidden connections and relations between real-world objects. Most contemporary research works employ state-of-the-art language models and techniques for determining the type of relationship existing between a pair of entities in a given sentence but are computationally expensive and fail to identify or match the entities present in the sentence as a single task, rather they breakdown the problem into subtasks or rely on a multi-module framework requiring multiple propagations through the network. The paper presents a novel methodology for extracting relations between multiple pairs of entities present in the sentence and performs the relation classification task. The proposed methodology employs a graph neural network, and in contrast to the existing research, the proposed mechanism makes use of a hybrid attention mechanism to dynamically optimize the graph edges to capture the relevant details in the network graph to aid as an attention mechanism to aid in faster computation, compared to the more typical transformer-based networks that overwhelm CPU-based systems. The paper also studies the effect of the number of hop transformations on the graph and other hyper-parameters controlling the input sentence representation. The proposed model architecture achieves a macro avg. F1 score of 86.2 on the SemEval 2010 relation extraction dataset, with further room for improvement.

A Span-based Multi-Modal Attention Network for joint entity-relation extraction

Joint extraction of entities and their relations not only depends on entity semantics but also highly correlates with contextual information and entity types. Therefore, an effective joint modelling method designed for handling information from different modalities can lead to a superior performance of the joint entity and relation extraction. Previous span-based models tended to focus on the internal semantics of a span but failed to effectively capture the interactions between the span and other modal information (such as tokens or labels). In this study, a Span-based Multi-Modal Attention Network (SMAN) is proposed for joint entity and relation extraction. The network introduces a cloze mechanism to simultaneously extract the context and span position information, and jointly models the span and label in the relation extraction stage. To determine the fine-grained associations between different modalities, a Modal-Enhanced Attention (MEA) module with two modes is designed and adopted in the modelling process. Experimental results reveal that the proposed model consistently outperforms the state-of-the-art for both entity recognition and relation extraction on the SciERC and ADE datasets, and beats other competing approaches by more than 1.42% F1 score for relation extraction on the CoNLL04 dataset. Extensive additional experiments further verify the effectiveness of the proposed model.

Digital surveillance in Latin American diseases outbreaks: information extraction from a novel Spanish corpus

BackgroundIn order to detect threats to public health and to be well-prepared for endemic and pandemic illness outbreaks, countries usually rely on event-based surveillance (EBS) and indicator-based surveillance systems. Event-based surveillance systems are key components of early warning systems and focus on fast capturing of data to detect threat signals through channels other than traditional surveillance. In this study, we develop Natural Language Processing tools that can be used within EBS systems. In particular, we focus on information extraction techniques that enable digital surveillance to monitor Internet data and social media.ResultsWe created an annotated Spanish corpus from ProMED-mail health reports regarding disease outbreaks in Latin America. The corpus has been used to train algorithms for two information extraction tasks: named entity recognition and relation extraction. The algorithms, based on deep learning and rules, have been applied to recognize diseases, hosts, and geographical locations where a disease is occurring, among other entities and relations. In addition, an in-depth analysis of micro-average F1 metrics shows the suitability of our approaches for both tasks.ConclusionsThe annotated corpus and algorithms presented could leverage the development of automated tools for extracting information from news and health reports written in Spanish. Moreover, this framework could be useful within EBS systems to support the early detection of Latin American disease outbreaks.

JCBIE: a joint continual learning neural network for biomedical information extraction

Extracting knowledge from heterogeneous data sources is fundamental for the construction of structured biomedical knowledge graphs (BKGs), where entities and relations are represented as nodes and edges in the graphs, respectively. Previous biomedical knowledge extraction methods simply considered limited entity types and relations by using a task-specific training set, which is insufficient for large-scale BKGs development and downstream task applications in different scenarios. To alleviate this issue, we propose a joint continual learning biomedical information extraction (JCBIE) network to extract entities and relations from different biomedical information datasets. By empirically studying different joint learning and continual learning strategies, the proposed JCBIE can learn and expand different types of entities and relations from different datasets. JCBIE uses two separated encoders in joint-feature extraction, hence can effectively avoid the feature confusion problem comparing with using one hard-parameter sharing encoder. Specifically, it allows us to adopt entity augmented inputs to establish the interaction between named entity recognition and relation extraction. Finally, a novel evaluation mechanism is proposed for measuring cross-corpus generalization errors, which was ignored by traditional evaluation methods. Our empirical studies show that JCBIE achieves promising performance when continual learning strategy is adopted with multiple corpora.

Towards Domain-Specific Knowledge Graph Construction for Flight Control Aided Maintenance

Flight control is a key system of modern aircraft. During each flight, pilots use flight control to control the forces of flight and also the aircraft’s direction and attitude. Whether flight control can work properly is closely related to safety such that daily maintenance is an essential task of airlines. Flight control maintenance heavily relies on expert knowledge. To facilitate knowledge achievement, aircraft manufacturers and airlines normally provide structural manuals for consulting. On the other hand, computer-aided maintenance systems are adopted for improving daily maintenance efficiency. However, we find that grass-roots engineers of airlines still inevitably consult unstructured technical manuals from time to time, for example, when meeting an unusual problem or an unfamiliar type of aircraft. Achieving effective knowledge from unstructured data is inefficient and inconvenient. Aiming at the problem, we propose a knowledge-graph-based maintenance prototype system as a complementary solution. The knowledge graph we built is dedicated for unstructured manuals referring to flight control. We first build ontology to represent key concepts and relation types and then perform entity-relation extraction adopting a pipeline paradigm with natural language processing techniques. To fully utilize domain-specific features, we present a hybrid method consisting of dedicated rules and a machine learning model for entity recognition. As for relation extraction, we leverage a two-stage Bi-LSTM (bi-directional long short-term memory networks) based method to improve the extraction precision by solving a sample imbalanced problem. We conduct comprehensive experiments to study the technical feasibility on real manuals from airlines. The average precision of entity recognition reaches 85%, and the average precision of relation extraction comes to 61%. Finally, we design a flight control maintenance prototype system based on the knowledge graph constructed and a graph database Neo4j. The prototype system takes alarm messages represented in natural language as the input and returns maintenance suggestions to serve grass-roots engineers.

A Novel Threat Intelligence Information Extraction System Combining Multiple Models

The increasing number of cyberattacks has made the cybersecurity situation more serious. Thus, it is urgent to use cyber threat intelligence to deal with the complex and changing cyber environment. However, cyber threat intelligence usually exists in an unstructured form, and a huge amount of data poses a great challenge to security analysts. To this end, this paper proposes a novel threat intelligence information extraction system combining multiple models, which contains four key steps: entity extraction, coreference resolution, relation extraction, and knowledge graph construction. In the entity extraction task, a multihead self-attention mechanism is adopted to extract the dependency relationships between words. In the coreference resolution task, contextual information and mention embedding are fused to improve the mention representation. Meanwhile, features of different dimensions are extracted using a convolutional neural network. In the relation extraction task, additional features such as part of speech, mention width, entity type, and distance of entity pairs are incorporated to improve the embedding representation. Finally, a knowledge graph is constructed to explicitly present entities and their relationships. Experimental results indicate that compared with the baseline model, the F1 score of our model is improved by at least 8.87, 9.82, and 10.56 on entity extraction, coreference resolution, and relation extraction, respectively. The knowledge graph in Neo4j demonstrates the effectiveness of our system.

The Joint Method of Triple Attention and Novel Loss Function for Entity Relation Extraction in Small Data-Driven Computational Social Systems

With the development of the social Internet of Things (IoT) and multimedia communications, our daily lives in computational social systems have become more convenient; for example, we can share shopping experiences and ask questions of people in an ad hoc network. Relation extraction focuses on supervised learning with adequate training data, and it helps to understand the knowledge behind the observed information. However, if only some social data in an unknown area can be used, how to obtain the related knowledge and information is a key topic for supporting social intelligence. This article proposes the joint method of triple attention and novel loss function for entity relation extraction by few-shot learning in computational social systems. We consider using a prototypical network as the base model to acquire support set prototypes and to compare queries with the prototypes for classification. First, triple attention is employed to make the query instances and support set share interactive information in a global and instancewise manner, highlighting the important features. Second, we combine a weighted Euclidean distance function with a multilayer perceptron (MLP) to perform class matching, which maps the generated features to their proper classifications, emphasizing the prominent dimensions in the feature space and relieving data sparsity. Third, triplet loss and uniformity regularization are used to solve the inconsistency problem faced by the support set, where the features of the support set in the same class are often far apart in different characteristic dimensions. Finally, the experimental results demonstrate the improved performance of our model on the FewRel dataset.

Application of Knowledge Graph in Water Conservancy Education Resource Organization under the Background of Big Data

The key to improving the readability and usage of educational resources is their orderly arrangement and integration. Knowledge graphs, which are a large-scale form of knowledge engineering, are an effective tool for managing and organizing educational resources. The water conservancy’s educational big data is separated into three tiers of objectives–courses–knowledge units based on the connotation level of self-directed learning. Combined with the idea of Outcome-based Education(OBE), the goal-oriented knowledge graph structure of water conservancy disciplines and graph creation method is proposed. The focus is the error accumulation problem brought about by the traditional relational extraction method of Named Entity Recognition based on rules or sequence labeling. We first complete this objective, and then the relationship classification is performed according to the water conservancy disciplines entity and relations joint extraction (WDERJE) model, on which the prompt mechanism design is based. Think of the entity-relationship extraction task as a sequence-to-sequence generation task, and take the structured extraction language to unify the coding entity extraction and relationship extraction structures. The evaluation results of the WDERJE model show that the F_0.5 value of each entity extraction is above 0.76, and the cumulative extraction relationship triple is nearly 180,000. The graph fully optimizes the organization and management of water conservancy education resources and effectively improves the readability and utilization rate of water conservancy teaching resources.

Researches Advanced in the Development and Application of Information Extraction

Natural language processing (NLP) is an interdisciplinary subject of linguistics and computer science, which processes the language used for human communication into a machine language that can be understood by machines.The task of information extraction is to obtain target information accurately and quickly from a large amount of data and improve the utilization of information. With the rapid development of Internet applications, how to quickly and accurately analyze the really useful information in these text data is particularly critical and urgent. Information extraction has become an important branch of natural language processing. Thanks to the rapid development of deep learning, the performance of information extraction has made breakthroughs in recent years. In this paper, based on the detailed literature analysis, this paper first review the development of information extraction. Secondly, the research progress of key technologies of information extraction is summarized from four aspects: named entity recognition, anaphora resolution, relationship extraction and event extraction. Finally, we analyze some main problems of information extraction and predict the research trend of information extraction.

Information extraction from electronic medical documents: state of the art and future research directions.

In the medical field, a doctor must have a comprehensive knowledge by reading and writing narrative documents, and he is responsible for every decision he takes for patients. Unfortunately, it is very tiring to read all necessary information about drugs, diseases and patients due to the large amount of documents that are increasing every day. Consequently, so many medical errors can happen and even kill people. Likewise, there is such an important field that can handle this problem, which is the information extraction. There are several important tasks in this field to extract the important and desired information from unstructured text written in natural language. The main principal tasks are named entity recognition and relation extraction since they can structure the text by extracting the relevant information. However, in order to treat the narrative text we should use natural language processing techniques to extract useful information and features. In our paper, we introduce and discuss the several techniques and solutions used in these tasks. Furthermore, we outline the challenges in information extraction from medical documents. In our knowledge, this is the most comprehensive survey in the literature with an experimental analysis and a suggestion for some uncovered directions.

Spatial Relationship Extraction of Geographic Entities Based on BERT Model

Geographic entity relationship extraction from text is an important way to acquire geographic knowledge. Entity relations in Chinese text are difficult to discover because of implicit representations of relations between entities in Chinese text. Therefore, using existing pattern matching and machine learning methods to extract entity relations from Chinese text often has problems such as insufficient artificial features, poor generality, inability to resolve word polysemy, and difficulty in making full use of contextual information. However, deep learning methods can better solve the above problems. This paper takes the spatial relation between geographic entities as the main research object to build a geographic entity spatial relation corpus, and propose a geographic entity relationship extraction method that combines BERT and attention mechanism. The method is trained and tested on the corpus. The results show that the F1 value of the BERT-BiGRU-Attention model proposed in this paper reaches 85.2% on the test set, which is a great improvement in relation extraction ability compared with other baseline models. The geographic entity relationship extraction method based on the BERT model can effectively learn the context information in Chinese text sentences, improve the accuracy of relationship extraction, and provide support for geographic knowledge graph construction and geographic entity information retrieval.

BiodivNERE: Gold standard corpora for named entity recognition and relation extraction in the biodiversity domain

Biodiversity is the assortment of life on earth covering evolutionary, ecological, biological, and social forms. To preserve life in all its variety and richness, it is imperative to monitor the current state of biodiversity and its change over time and to understand the forces driving it. This need has resulted in numerous works being published in this field. With this, a large amount of textual data (publications) and metadata (e.g. dataset description) has been generated. To support the management and analysis of these data, two techniques from computer science are of interest, namely Named Entity Recognition (NER) and Relation Extraction (RE). While the former enables better content discovery and understanding, the latter fosters the analysis by detecting connections between entities and, thus, allows us to draw conclusions and answer relevant domain-specific questions. To automatically predict entities and their relations, machine/deep learning techniques could be used. The training and evaluation of those techniques require labelled corpora. In this paper, we present two gold-standard corpora for Named Entity Recognition (NER) and Relation Extraction (RE) generated from biodiversity datasets metadata and abstracts that can be used as evaluation benchmarks for the development of new computer-supported tools that require machine learning or deep learning techniques. These corpora are manually labelled and verified by biodiversity experts. In addition, we explain the detailed steps of constructing these datasets. Moreover, we demonstrate the underlying ontology for the classes and relations used to annotate such corpora.

Research on Intelligent Power Marketing Inspection Model Based on Knowledge Graph

The traditional topic-based auditing model lacks the ability of multi-object and multi-abnormal correlation auditing, which makes it impossible to solve the multi-scenario and multi-factor correlation-based auditing problem. This paper designs an intelligent power marketing audit model based on a knowledge graph. First, an entity identification and relationship extraction method for power marketing business based on NLP (natural language processing) and sequence annotation technology is proposed, and the description content is imported into the knowledge graph database; then, semantic disambiguation and knowledge are carried out by using bidirectional encoder representation from transformers (BERT). Link to build a knowledge map of business audit rules: Finally, an experimental analysis is carried out by taking the copying and receiving business with a large business volume in the marketing audit work as an example, and it is verified that the proposed model can effectively improve the information analysis ability and the audit accuracy of the audit work.

Entity relation extraction in the medical domain: based on data augmentation.

Entity relation extraction is an important task in the construction of professional knowledge graphs in the medical field. Research on entity relation extraction for academic books in the medical field has revealed that there is a great difference in the number of different entity relations, which has led to the formation of a typical unbalanced data set that is difficult to recognize but has certain research value. In this article, we propose a new entity relation extraction method based on data augmentation. According to the distribution of individual entity relation classes in the data set, the probability of whether a text is augmented during training was calculated. In text-oriented data augmentation, different augmentation methods perform differently in different language environments. The reinforcement of learning determines which data augmentation method to use in the current language environment. This strategy was applied to the entity relation extraction of the medical professional book, Pharmacopoeia of the People's Republic of China, and different data augmentation methods (i.e., no data augmentation, traditional data augmentation, and reinforcement learning-based data augmentation) were compared under the same neural network model. The deep-learning model using data augmentation was better than the model without data augmentation, as data augmentation significantly improved the evaluation indicators of the relation classes with low data volumes in the unbalanced data set and slightly improved the evaluation indicators of the relation classes with sufficient features and large data volumes. Additionally, the deep-learning model using reinforcement learning-based data augmentation was superior to the deep-learning model using traditional data augmentation. We found that after the application of reinforcement learning-based data augmentation, the evaluation indicators of the multiple relation classes were much better than those to which reinforcement learning-based data augmentation had not been applied. For unbalanced data sets, data augmentation can effectively improve the ability of the deep-learning model to obtain data features, and reinforcement learning-based data augmentation can further enhance this ability. Our experiments confirmed the superiority of reinforcement learning-based data augmentation.

Research on Chinese Medical Entity Relation Extraction Based on Syntactic Dependency Structure Information

Extracting entity relations from unstructured medical texts is a fundamental task in the field of medical information extraction. In relation extraction, dependency trees contain rich structural information that helps capture the long-range relations between entities. However, many models cannot effectively use dependency information or learn sentence information adequately. In this paper, we propose a relation extraction model based on syntactic dependency structure information. First, the model learns sentence sequence information by Bi-LSTM. Then, the model learns syntactic dependency structure information through graph convolutional networks. Meanwhile, in order to remove irrelevant information from the dependencies, the model adopts a new pruning strategy. Finally, the model adds a multi-head attention mechanism to focus on the entity information in the sentence from multiple aspects. We evaluate the proposed model on a Chinese medical entity relation extraction dataset. Experimental results show that our model can learn dependency relation information better and has higher performance than other baseline models.

A Survey of Information Extraction Based on Deep Learning

As a core task and an important link in the fields of natural language understanding and information retrieval, information extraction (IE) can structure and semanticize unstructured multi-modal information. In recent years, deep learning (DL) has attracted considerable research attention to IE tasks. Deep learning-based entity relation extraction techniques have gradually surpassed traditional feature- and kernel-function-based methods in terms of the depth of feature extraction and model accuracy. In this paper, we explain the basic concepts of IE and DL, primarily expounding on the research progress and achievements of DL technologies in the field of IE. At the level of IE tasks, it is expounded from entity relationship extraction, event extraction, and multi-modal information extraction three aspects, and creates a comparative analysis of various extraction techniques. We also summarize the prospects and development trends in DL in the field of IE as well as difficulties requiring further study. It is believed that research can be carried out in the direction of multi-model and multi-task joint extraction, information extraction based on knowledge enhancement, and information fusion based on multi-modal at the method level. At the model level, further research should be carried out in the aspects of strengthening theoretical research, model lightweight, and improving model generalization ability.