Entity Relation Extraction Research Articles

Deep learning joint models for extracting entities and relations in biomedical: a survey and comparison.

The rapid development of biomedicine has produced a large number of biomedical written materials. These unstructured text data create serious challenges for biomedical researchers to find information. Biomedical named entity recognition (BioNER) and biomedical relation extraction (BioRE) are the two most fundamental tasks of biomedical text mining. Accurately and efficiently identifying entities and extracting relations have become very important. Methods that perform two tasks separately are called pipeline models, and they have shortcomings such as insufficient interaction, low extraction quality and easy redundancy. To overcome the above shortcomings, many deep learning-based joint name entity recognition and relation extraction models have been proposed, and they have achieved advanced performance. This paper comprehensively summarize deep learning models for joint name entity recognition and relation extraction for biomedicine. The joint BioNER and BioRE models are discussed in the light of the challenges existing in the BioNER and BioRE tasks. Five joint BioNER and BioRE models and one pipeline model are selected for comparative experiments on four biomedical public datasets, and the experimental results are analyzed. Finally, we discuss the opportunities for future development of deep learning-based joint BioNER and BioRE models.

A Deep Learning based Feature Entity Relationship Extraction Method for Telemedicine Sensing Big Data

To solve the problem of inaccurate entity extraction caused by low application efficiency and big data noise in telemedicine sensing data, a deep learning-based method for entity relationship extraction in telemedicine big data is proposed. By analyzing the distribution structure of the medical sensing big data, the fuzzy function of the distribution shape is calculated and the seed relationship set is transformed by the inverse Shearlet transform. Combined with the deep learning technology, the GMM-GAN data enhancement model is built, the interactive medical sensing big data features are obtained, the association rules are matched one by one, the noiseless medical sensing data are extracted in time sequence, the feature items with the highest similarity are obtained and used as the constraint to complete the feature entity relationship extraction of the medical sensing data. The experimental results show that the extracted similarity of entity relations is more than 70%, which can handle overly long and complex sentences in telemedicine information text; the extraction time is the shortest and the volatility is low.

Fine-Grained Drug Interaction Extraction Based on Entity Pair Calibration and Pre-Training Model for Chinese Drug Instructions

Existing pharmaceutical information extraction research often focus on standalone entity or relationship identification tasks over drug instructions. There is a lack of a holistic solution for drug knowledge extraction. Moreover, current methods perform poorly in extracting fine-grained interaction relations from drug instructions. To solve these problems, this paper proposes an information extraction framework for drug instructions. The framework proposes deep learning models with fine-tuned pre-training models for entity recognition and relation extraction. In addition, it incorporates an novel entity pair calibration process to promote the performance for fine-grained relation extraction. The framework experiments on more than 60k Chinese drug description sentences from 4000 drug instructions. Empirical results show that the framework can successfully identify drug related entities (F1 3 0.95) and their relations (F1 3 0.83) from the realistic dataset, and the entity pair calibration plays an important role (~5% F1 score improvement) in extracting fine-grained relations.

Knowledge-driven recognition methodology for electricity safety hazard scenarios

This study proposes a knowledge-driven recognition methodology for electrical safety hazard scenarios to address the inability of maintenance personnel to accurately identify safety hazards and deal with them effectively due to complicated electrical safety hazard scenarios. First, a BERT-BiLSTM-CRF-based safety hazard entity recognition model and ResPCNN-ATT-based safety hazard relation extraction model are designed. The entity recognition and relation extraction in electricity safety hazard records are implemented by them. Then, the knowledge graph of the electricity safety hazard domain is constructed, and the semantic information of scenes provided in the hazard records is incorporated into the relationship constraints among various entities. Finally, an intelligent reasoning model of hazard scenarios based on a knowledge graph of electricity safety hazards is proposed. The hazard scenes and solution measures can be quickly located based on the types of hazard equipment and hazardous phenomena. The proposed approach’s effectiveness and engineering practicality was verified using the accumulated hazard records of a provincial power company.

A Relation-Oriented Model With Global Context Information for Joint Extraction of Overlapping Relations and Entities.

The entity relation extraction in the form of triples from unstructured text is a key step for self-learning knowledge graph construction. Two main methods have been proposed to extract relation triples, namely, the pipeline method and the joint learning approach. However, these models do not deal with the overlapping relation problem well. To overcome this challenge, we present a relation-oriented model with global context information for joint entity relation extraction, namely, ROMGCJE, which is an encoder–decoder model. The encoder layer aims to build long-term dependencies among words and capture rich global context representation. Besides, the relation-aware attention mechanism is applied to make use of the relation information to guide the entity detection. The decoder part consists of a multi-relation classifier for the relation classification task, and an improved long short-term memory for the entity recognition task. Finally, the minimum risk training mechanism is introduced to jointly train the model to generate final relation triples. Comprehensive experiments conducted on two public datasets, NYT and WebNLG, show that our model can effectively extract overlapping relation triples and outperforms the current state-of-the-art methods.

A Joint Learning Model to Extract Entities and Relations for Chinese Literature Based on Self-Attention

Extracting structured information from massive and heterogeneous text is a hot research topic in the field of natural language processing. It includes two key technologies: named entity recognition (NER) and relation extraction (RE). However, previous NER models consider less about the influence of mutual attention between words in the text on the prediction of entity labels, and there is less research on how to more fully extract sentence information for relational classification. In addition, previous research treats NER and RE as a pipeline of two separated tasks, which neglects the connection between them, and is mainly focused on the English corpus. In this paper, based on the self-attention mechanism, bidirectional long short-term memory (BiLSTM) neural network and conditional random field (CRF) model, we put forth a Chinese NER method based on BiLSTM-Self-Attention-CRF and a RE method based on BiLSTM-Multilevel-Attention in the field of Chinese literature. In particular, considering the relationship between these two tasks in terms of word vector and context feature representation in the neural network model, we put forth a joint learning method for NER and RE tasks based on the same underlying module, which jointly updates the parameters of the shared module during the training of these two tasks. For performance evaluation, we make use of the largest Chinese data set containing these two tasks. Experimental results show that the proposed independently trained NER and RE models achieve better performance than all previous methods, and our joint NER-RE training model outperforms the independently-trained NER and RE model.

Information Extraction and Knowledge Graph Construction for Enterprises in China’s Free Trade Zone

With the establishment and rapid development of the China (Shanghai) Pilot Free Trade Zone (FTZ), the scale of enterprises in the zone has grown rapidly. This paper takes the actual needs of the Shanghai FTZ as the background, extracts key information such as entities from the Big Data related to Internet enterprises, further constructs the enterprise knowledge graph, and applies it to the supervision and service of the FTZ. The enterprise knowledge graph is constructed using the Neo4j graph database. To verify the effectiveness of the named entity identification and relationship extraction methods proposed in this paper, experiments were conducted to validate them, and both achieved good results.

Discovering Content through Text Mining for a Synthetic Biology Knowledge System.

Scientific articles contain a wealth of information about experimental methods and results describing biological designs. Due to its unstructured nature and multiple sources of ambiguity and variability, extracting this information from text is a difficult task. In this paper, we describe the development of the synthetic biology knowledge system (SBKS) text processing pipeline. The pipeline uses natural language processing techniques to extract and correlate information from the literature for synthetic biology researchers. Specifically, we apply named entity recognition, relation extraction, concept grounding, and topic modeling to extract information from published literature to link articles to elements within our knowledge system. Our results show the efficacy of each of the components on synthetic biology literature and provide future directions for further advancement of the pipeline.

Relationship classification based on dependency parsing and the pretraining model

As an important part of information extraction, relationship extraction aims to extract the relationships between given entities from natural language text. Based on the pretraining model R-BERT, this paper proposes an entity relationship extraction method that integrates an entity dependency path and pretraining model, which generates a dependency parse tree by dependency parsing, obtains the dependency path of an entity pair via a given entity, and uses an entity dependency path to exclude information such as modifier chunks and useless entities in sentences. This model has achieved good F1 value performance on the SemEval2010 Task 8 dataset. Experiments on datasets show that dependency parsing can provide context information for models and improve performance.

Natural language processing-optimized case selection for real-world evidence studies.

1556 Background: Much information describing a patient’s cancer treatment remains in unstructured text in electronic health records and is not recorded in discrete data fields. Accurate data completeness is essential for quality care improvement and research studies on de-identified patient records. Accessing this high-value content often requires manual and extensive curation review. Methods: AstraZeneca, CancerLinQ, ConcertAI, and Tempus have developed a natural language processing (NLP)-assisted process to improve clinical cohort selection for targeted curation efforts. Hybrid, machine-learning model development included text classification, named entity recognition, relation extraction and false positive removal. A subset of nearly 60,000 lung cancer cases were included from the CancerLinQ database, comprised of multiple source EHR systems. NLP models extracted EGFR status, stage, histology, radiation therapy, surgical resection and oral medications. Based on the results, cases were selected for additional manual curation, where curators confirmed findings of the NLP-processed data. Results: NLP methods improved cohort identification. Successfully returned cases using the NLP method ranged from 75.2% to 96.5% over more general case selection criteria based on limited structured data. For all cohorts combined, 84.2% of the cases sent out for NLP curation were returned with curated content (Table). Each cohort contained a range of NLP-derived elements for curators to further review. In comparison, more general case selection criteria yielded a total of 3,878 cases returned out of 41,186 lung cancer cases sent for curation, for a success rate of only 9.6%. Conclusions: NLP-driven case selection of six distinct, complex lung cohorts resulted in an order of magnitude improvement in eligibility over candidate selection using structured EHR data alone. This study demonstrates NLP-assisted approaches can significantly improve efficiency in curating unstructured health data. [Table: see text]

Medical Knowledge Graph to Promote Rational Drug Use: Model Development and Performance Evaluation

Knowledge Graph (KG) has been proven effective in representing and modeling structured information, especially in the medical domain. However, obtaining structured medical information usually depends on the manual processing of medical experts. Meanwhile, the construction of Medical Knowledge Graph (MKG) remains a crucial problem in medical informatization. This work presents a novel method for constructing MKGto drive the application of Rational Drug Use (RDU). We first collect and preprocess the corpora from various types of resources, and then develop a medical ontology via studying the concepts in RDUdomain, authoritative books and drug instructions. Based on the medical ontology, we formulate a scheme to annotate the corpora and construct the dataset for extracting entities and relations. We utilize two mechanisms to extract entities and relations respectively. The former is based on deep learning, while the latter is the rule-based method. In the last stage, we disambiguate and standardize the results of entity relation extraction to construct and enrich the MKG. The experimental results verify the effectiveness of the proposed methods.

Plant phenotype relationship corpus for biomedical relationships between plants and phenotypes

Medicinal plants have demonstrated therapeutic potential for applicability for a wide range of observable characteristics in the human body, known as “phenotype,” and have been considered favorably in clinical treatment. With an ever increasing interest in plants, many researchers have attempted to extract meaningful information by identifying relationships between plants and phenotypes from the existing literature. Although natural language processing (NLP) aims to extract useful information from unstructured textual data, there is no appropriate corpus available to train and evaluate the NLP model for plants and phenotypes. Therefore, in the present study, we have presented the plant-phenotype relationship (PPR) corpus, a high-quality resource that supports the development of various NLP fields; it includes information derived from 600 PubMed abstracts corresponding to 5,668 plant and 11,282 phenotype entities, and demonstrates a total of 9,709 relationships. We have also described benchmark results through named entity recognition and relation extraction systems to verify the quality of our data and to show the significant performance of NLP tasks in the PPR test set.

Vulcan: Automatic extraction and analysis of cyber threat intelligence from unstructured text

To counteract the rapidly evolving cyber threats, many research efforts have been made to design cyber threat intelligence (CTI) systems that extract CTI data from publicly available sources. Specifically, indicators of compromise (IOC), such as file hash and IP address, receives the most attention among security researchers. However, the ability of IOC-centric CTI systems to understand and detect threats remains questionable for two reasons. First, IOCs are forensic artifacts that indicate that an endpoint or network has been compromised. They cannot depict the technical details of threats. Second, attackers frequently change infrastructure and static indicators, which makes IOCs have a very short lifespan. Therefore, when designing a CTI system, we should turn our attention to other types of CTI data that are helpful in threat understanding and detection (e.g., attack vector, tool).In this work, we propose Vulcan, a novel CTI system that extracts descriptive or static CTI data from unstructured text and determines their semantic relationships. To do this, we design a neural language model-based named entity recognition (NER) and relation extraction (RE) models tailored for cybersecurity domain. The experimental results confirm that Vulcan is highly accurate with an average F1-score of 0.972 and 0.985 for NER and RE tasks, respectively. Vulcan also provides an environment where security practitioners can develop applications for threat analysis. To prove the applicability of Vulcan, we introduce two applications, evolution identification and threat profiling. The applications save time and labor costs to analyze cyber threats and show the detailed characteristics of the threats.

Construction and Application of Text Entity Relation Joint Extraction Model Based on Multi-Head Attention Neural Network.

Entity relationship extraction is one of the key areas of information extraction and is an important research content in the field of natural language processing. Based on past research, this paper proposes a combined extraction model based on a multi-headed attention neural network. Based on the BERT training model architecture, this paper extracts textual entities and relations tasks. At the same time, it integrates the naming entity feature, the terminology labeling characteristics, and the training relationship. The multi-attention mechanism and improved neural structures are added to the model to enhance the characteristic extraction capacity of the model. By studying the parameters of the multi-head attention mechanism, it is shown that the optimal parameters of the multi-head attention are h = 8, dv = 16, and the classification effect of the model is the best at this time. After experimental analysis, comparing the traditional text entity relationship extraction model and the multi-head attention neural network joint extraction model, the model entity relationship extraction effect was evaluated from the aspects of comprehensive evaluation index F1, accuracy rate P, and system time consumed. Experiments show: First, in the accuracy indicator, Xception performance is best, reaching 87.7%, indicating that the model extraction feature effect is enhanced. Second, with the increase of the number of iterative times, the verification set curve and the training set curve have increased to 96% and 98%, respectively, and the model has a strong generalization ability. Third, the model completes the extraction of all data in the test set in 1005 ms, which is an acceptable speed. Therefore, the model test results in this article are good, with a strong practical value.

DTGCN: a method combining dependency tree and graph convolutional networks for Chinese long-interval named entity relationship extraction

DTGCN: a method combining dependency tree and graph convolutional networks for Chinese long-interval named entity relationship extraction

Extraction of entity relations from Chinese medical literature based on multi-scale CRNN.

BackgroundEntity relation extraction technology can be used to extract entities and relations from medical literature, and automatically establish professional mapping knowledge domains. The classical text classification model, convolutional neural networks for sentence classification (TEXTCNN), has been shown to have good classification performance, but also has a long-distance dependency problem, which is a common problem of convolutional neural networks (CNNs). Recurrent neural networks (RNN) address the long-distance dependency problem but cannot capture text features at a specific scale in the text.MethodsTo solve these problems, this study sought to establish a model with a multi-scale convolutional recurrent neural network for Sentence Classification (TEXTCRNN) to address the deficiencies in the 2 neural network structures. In entity relation extraction, the entity pair is generally composed of a subject and an object, but as the subject in the entity pair of medical literature is always omitted, it is difficult to use this coding method to obtain general entity position information. Thus, we proposed a new coding method to obtain entity position information to re-establish the relationship between subject and object and complete the entity relation extraction.ResultsBy comparing the benchmark neural network model and 2 typical multi-scale TEXTCRNN models, the TEXTCRNN [bidirectional long- and short-term memory (BiLSTM)] and TEXTCRNN [double-layer stacking gated recurrent unit (GRU)], the results showed that the multi-scale CRNN model had the best F1 value performance, and the TEXTCRNN (double-layer stacking GRU) was more capable of entity relation classification when the same entity word did not belong to the same entity relation.ConclusionsThe experimental results of the entity relation extraction from Pharmacopoeia of the People’s Republic of China—Guidelines for Clinical Drug Use—Volume of Chemical Drugs and Biological Products showed that entity relation extraction could effectively proceed using the new labeling method. Additionally, compared to typical neural network models, including the TEXTCNN, GRU, and BiLSTM, the multi-scale convolutional recurrent neural network structure had advantages across several evaluation indicators.

Connecting the dots in clinical document understanding with Relation Extraction at scale

<h2>Abstract</h2> We present a text mining framework based on top of the Spark NLP library - comprising of Named Entity Recognition (NER) and Relation Extraction (RE) models, which expands on previous work in three main ways. First, we release new RE model architectures that obtain state-of-the-art F1 scores on 5 out of 7 benchmark datasets. Second, we introduce a modular approach to train and stack multiple models in a single nlp pipeline in a production grade library with little coding. Third, we apply these models in practical applications including knowledge graph generation, prescription parsing, and robust ontology mapping.

An Entity Relationship Extraction Model Based on BERT-BLSTM-CRF for Food Safety Domain.

Dealing with food safety issues in time through online public opinion incidents can reduce the impact of incidents and protect human health effectively. Therefore, by the smart technology of extracting the entity relationship of public opinion events in the food field, the knowledge graph of the food safety field is constructed to discover the relationship between food safety issues. To solve the problem of multi-entity relationships in food safety incident sentences for few-shot learning, this paper adopts the pipeline-type extraction method. Entity relationship is extracted from Bidirectional Encoder Representation from Transformers (BERTs) joined Bidirectional Long Short-Term Memory (BLSTM), namely, the BERT-BLSTM network model. Based on the entity relationship types extracted from the BERT-BLSTM model and the introduction of Chinese character features, an entity pair extraction model based on the BERT-BLSTM-conditional random field (CRF) is established. In this paper, several common deep neural network models are compared with the BERT-BLSTM-CRF model with a food public opinion events dataset. Experimental results show that the precision of the entity relationship extraction model based on BERT-BLSTM-CRF is 3.29%∼23.25% higher than that of other models in the food public opinion events dataset, which verifies the validity and rationality of the model proposed in this paper.

A pattern-first pipeline approach for entity and relation extraction

Entity-relation extraction is the task of extracting entities and their semantic relations from a piece of unstructured text. In recent studies, Machine Reading Comprehension (MRC) based methods have been applied to this task and achieved significant results. As a pipelined approach, these methods always extract head entities first, and then identify related tail entities by enumerating each relationship. These entity-first methods will lead to the entity redundancy problem. They also suffer from the error propagation issue, which is an inherent issue of the multi-step inference process. Moreover, most existing MRC-based models, which use tagging-based methods for entity recognition, could not deal with overlapping entities. To address these, we propose Patti, a Pattern-First Pipeline Approach for Entity and Relation Extraction. Firstly, Patti leverages a novel MRC-based pattern classifier to identify relation patterns. Next, a span-based method was introduced to extract entities under the guidance of questions parameterized by the patterns yield in the first step. Finally, to alleviate the error propagation issue, Patti employs an additional MRC-based classifier to remove falsely extracted candidate entity-relation triples. Experiment results show that our approach significantly outperforms the entity-first baseline models on CoNLL04 and ACE05 datasets.

Joint Extraction of Long-Distance Entity Relation by Aggregating Local- and Semantic-Dependent Features

The relations among long-distance entities are important for text understanding. Long-distance entities mean a pair of entities separated by more words in the text. Most current work of the joint extraction of entity relation is between entities that are close together. These methods are not suitable for long-distance entity relation extraction for they usually ignore the interaction between long-distance entities. In this paper, we propose Long-Distance Feature Aggregation Model (LDFAM), an end-to-end long-distance entity relation joint extraction model, which uses a Bi-GRU and weighted graph neural network to model text and learn its local- and semantic-dependent features. The long-distance dependence between entities is obtained by aggregating these two types of features. For entity recognition, we use conditional random fields to further obtain the dependency information between tags. For relation extraction, we fuse entity tag information for biaffine pairwise scoring. Experimental results on NYT and SemEval2010-Task8 datasets show that our model has achieved significant improvements compared to the baseline model. Finally, we apply the model to the field of material science to extract entity relationship knowledge from material literature. Experimental results show that benefiting from the long-distance entity relation extraction, the proposed method can get more useful knowledge from material literature and improve the knowledge coverage of material domain knowledge acquisition.