Entity Relation Extraction Research Articles

Contrastive Information Extraction With Generative Transformer

Information extraction tasks such as entity relation extraction and event extraction are of great importance for natural language processing and knowledge graph construction. In this paper, we revisit the end-to-end information extraction task for sequence generation. Since generative information extraction may struggle to capture long-term dependencies and generate unfaithful triples, we introduce a novel model, contrastive information extraction with a generative transformer. Specifically, we introduce a single shared transformer module for an encoder-decoder-based generation. To generate faithful results, we propose a novel triplet contrastive training object. Moreover, we introduce two mechanisms to further improve model performance (i.e., batch-wise dynamic attention-masking and triple-wise calibration). Experimental results on five datasets (i.e., NYT, WebNLG, MIE, ACE-2005, and MUC-4) show that our approach achieves better performance than baselines.1

DKG-PIPD: A Novel Method About Building Deep Knowledge Graph

In this study, a method about building Deep Knowledge Graph for the Plant Insect Pest and Disease, namely DKG-PIPD, was proposed. Specifically, the semi-automatic extraction of semi-structured and unstructured knowledge was carried out on the basis of domain ontology, and the knowledge graph was stored in the third-party knowledge database according to the corpus characteristic of the plant insect pest and disease, to realize the visual display of entity interactive relationship and knowledge inference. Furthermore, DKG-PIPD performed joint extraction about the entity and the relationship in unstructured knowledge in a corpus tagging method that is suitable for domain data. In this way, the entity and the relationship were annotated synchronically, and the triplet can be obtained directly through label matching and label mapping, which not only effectively improved the annotation efficiency, but also solved the problem of one-versus-many overlapping relation extraction. In addition, DKG-PIPD used a novel end-to-end model to train and predict on the crawled dataset. The experimental contrast results with other classical benchmark methods demonstrated the effectiveness of the proposed method. Moreover, the related work in this paper first introduced the general architecture required for the building of knowledge graph, and then summarized its key points, that is, named entity recognition, entity relationship extraction and knowledge inference using deep learning are emphatically introduced. Finally, the improvement direction of this paper was also introduced in the discussion section.

BERT based clinical knowledge extraction for biomedical knowledge graph construction and analysis

Background: Knowledge is evolving over time, often as a result of new discoveries or changes in the adopted methods of reasoning. Also, new facts or evidence may become available, leading to new understandings of complex phenomena. This is particularly true in the biomedical field, where scientists and physicians are constantly striving to find new methods of diagnosis, treatment and eventually cure. Knowledge Graphs (KGs) offer a real way of organizing and retrieving the massive and growing amount of biomedical knowledge.Objective: We propose an end-to-end approach for knowledge extraction and analysis from biomedical clinical notes using the Bidirectional Encoder Representations from Transformers (BERT) model and Conditional Random Field (CRF) layer.Methods: The approach is based on knowledge graphs, which can effectively process abstract biomedical concepts such as relationships and interactions between medical entities. Besides offering an intuitive way to visualize these concepts, KGs can solve more complex knowledge retrieval problems by simplifying them into simpler representations or by transforming the problems into representations from different perspectives. We created a biomedical Knowledge Graph using using Natural Language Processing models for named entity recognition and relation extraction. The generated biomedical knowledge graphs (KGs) are then used for question answering.Results: The proposed framework can successfully extract relevant structured information with high accuracy (90.7% for Named-entity recognition (NER), 88% for relation extraction (RE)), according to experimental findings based on real-world 505 patient biomedical unstructured clinical notes.Conclusions:In this paper, we propose a novel end-to-end system for the construction of a biomedical knowledge graph from clinical textual using a variation of BERT models.

Construction and Application of Knowledge Graph for Quality and Safety Supervision of Transportation Engineering

Knowledge graph technology play a more and more important role in various fields of industry and academia. This paper firstly introduces the general framework of the knowledge graph construction, which includes three stages: information extraction, knowledge fusion and knowledge processing. In order to improve the efficiency of quality and safety supervision of transportation engineering construction, this paper constructs a knowledge graph by acquiring multi-sources heterogeneous data from supervision of transportation engineering quality and safety. It employs a bottom-up construction strategy and some natural language processing methods to solve the problems of the knowledge extraction for transportation engineering construction. We use the entity relation extraction method to extract the entity triples from the multi-sources heterogeneous data, and then employ knowledge inference to complete the edges in the constructed knowledge graph, finally perform quality evaluation to add the valid triples to the knowledge graph for updating. Subgraph matching technology is also exploited to retrieve the constructed knowledge graph for efficiently acquiring the useful knowledge about the quality and safety of transportation engineering projects. The results show that the constructed knowledge graph provides a practical and valuable tool for the quality and safety supervision of transportation engineering construction.

A Hybrid Attention Mechanism for Multi-target Entity Relation Extraction using Graph Neural Networks

A Hybrid Attention Mechanism for Multi-target Entity Relation Extraction using Graph Neural Networks

A Graph Convolutional Network With Multiple Dependency Representations for Relation Extraction

Dependency analysis can assist neural networks to capture semantic features within a sentence for entity relation extraction (RE). Both hard and soft strategies of encoding dependency tree structure have been developed to balance the beneficial extra information against the unfavorable interference in the task of RE. A wide application of graph convolutional network (GCN) in the field of natural language processing (NLP) has demonstrated its effectiveness in encoding the input sentence with the dependency tree structure, as well as its efficiency in parallel computation. This study proposes a novel GCN-based model using multiple representations to depict the dependency tree from various perspectives, and combines those dependency representations afterward to obtain a better sentence representation for relation classification. This model can maximally draw from the sentence the semantic features relevant to the relationship between entities. Results show that our model achieves state-of-the-art performance in terms of the F <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> score (68.0) on the Text Analysis Conference relation extraction dataset (TACRED). In addition, we verify that the renormalization parameter in the GCN operation should be carefully chosen to help GCN-based models achieve its best performance.

A Novel Chinese Entity Relationship Extraction Method Based on the Bidirectional Maximum Entropy Markov Model

To identify relationships among entities in natural language texts, extraction of entity relationships technically provides a fundamental support for knowledge graph, intelligent information retrieval, and semantic analysis, promotes the construction of knowledge bases, and improves efficiency of searching and semantic analysis. Traditional methods of relationship extraction, either those proposed at the earlier times or those based on traditional machine learning and deep learning, have focused on keeping relationships and entities in their own silos: extracting relationships and entities are conducted in steps before obtaining the mappings. To address this problem, a novel Chinese relationship extraction method is proposed in this paper. Firstly, the triple is treated as an entity relation chain and can identify the entity before the relationship and predict its corresponding relationship and the entity after the relationship. Secondly, the Joint Extraction of Entity Mentions and Relations model is based on the Bidirectional Long Short‐Term Memory and Maximum Entropy Markov Model (Bi‐MEMM). Experimental results indicate that the proposed model can achieve a precision of 79.2% which is much higher than that of traditional models.

A span-graph neural model for overlapping entity relation extraction in biomedical texts.

Entity relation extraction is one of the fundamental tasks in biomedical text mining, which is usually solved by the models from natural language processing. Compared with traditional pipeline methods, joint methods can avoid the error propagation from entity to relation, giving better performances. However, the existing joint models are built upon sequential scheme, and fail to detect overlapping entity and relation, which are ubiquitous in biomedical texts. The main reason is that sequential models have relatively weaker power in capturing long-range dependencies, which results in lower performance in encoding longer sentences. In this article, we propose a novel span-graph neural model for jointly extracting overlapping entity relation in biomedical texts. Our model treats the task as relation triplets prediction, and builds the entity-graph by enumerating possible candidate entity spans. The proposed model captures the relationship between the correlated entities via a span scorer and a relation scorer, respectively, and finally outputs all valid relational triplets. Experimental results on two biomedical entity relation extraction tasks, including drug-drug interaction detection and protein-protein interaction detection, show that the proposed method outperforms previous models by a substantial margin, demonstrating the effectiveness of span-graph-based method for overlapping relation extraction in biomedical texts. Further in-depth analysis proves that our model is more effective in capturing the long-range dependencies for relation extraction compared with the sequential models. Related codes are made publicly available at http://github.com/Baxelyne/SpanBioER.

Joint Entity-Relation Extraction via Improved Graph Attention Networks

Joint named entity recognition and relation extraction is an essential natural language processing task that aims to identify entities and extract the corresponding relations in an end-to-end manner. At present, compared with the named entity recognition task, the relation extraction task performs poorly on complex text. To solve this problem, we proposed a novel joint model named extracting Entity-Relations viaImproved Graph Attention networks (ERIGAT), which enhances the ability of the relation extraction task. In our proposed model, we introduced the graph attention network to extract entities and relations after graph embedding based on constructing symmetry relations. To mitigate the over-smoothing problem of graph convolutional networks, inspired by matrix factorization, we improved the graph attention network by designing a new multi-head attention mechanism and sharing attention parameters. To enhance the model robustness, we adopted the adversarial training to generate adversarial samples for training by adding tiny perturbations. Comparing with typical baseline models, we comprehensively evaluated our model by conducting experiments on an open domain dataset (CoNLL04) and a medical domain dataset (ADE). The experimental results demonstrate the effectiveness of ERIGAT in extracting entity and relation information.

Entity–Relation Extraction—A Novel and Lightweight Method Based on a Gate Linear Mechanism

Entity–relation extraction has attracted considerable attention in recent years as a fundamental task in natural language processing. The goal of entity–relation extraction is to discover the relation structures of entities from a natural language sentence. Most existing models approach this task using recurrent neural nets (RNNs); however, given the sequential nature of RNNs, the states cannot be computed in parallel, which slows the machine comprehension. In this paper, we propose a new end-to-end model based on dilated convolutional units and the gate linear mechanism as an alternative to those recurrent models. We find that relation extraction becomes more difficult as the sentence length increases. In this paper, we introduce dynamic convolutions based on lightweight convolutions to process long sequences, which thus reduces the number of parameters to a low level. Another challenge in relation extraction is relation spans potentially overlapping in a sentence, representing a bottleneck for the detection of multiple relational triplets. To alleviate this problem, we design an entirely new prediction scheme to extract relational pairs and additionally boost performance. We conduct experiments on two widely used datasets, and the results show that our model outperforms the baselines by a large margin.

Information Extraction from Text Intensive and Visually Rich Banking Documents

Document types, where visual and textual information plays an important role in their analysis and understanding, pose a new and attractive area for information extraction research. Although cheques, invoices, and receipts have been studied in some previous multi-modal studies, banking documents present an unexplored area due to the naturalness of the text they possess in addition to their visual richness. This article presents the first study which uses visual and textual information for deep-learning based information extraction on text-intensive and visually rich scanned documents which are, in this instance, unstructured banking documents, or more precisely, money transfer orders. The impact of using different neural word representations (i.e., FastText, ELMo, and BERT) on IE subtasks (namely, named entity recognition and relation extraction stages), positional features of words on document images and auxiliary learning with some other tasks are investigated. The article proposes a new relation extraction algorithm based on graph factorization to solve the complex relation extraction problem where the relations within documents are n-ary, nested, document-level, and previously indeterminate in quantity. Our experiments revealed that the use of deep learning algorithms yielded around 10 percentage points improvement on the IE sub-tasks. The inclusion of word positional features yielded around 3 percentage points of improvement in some specific information fields. Similarly, our auxiliary learning experiments yielded around 2 percentage points of improvement on some information fields associated with the specific transaction type detected by our auxiliary task. The integration of the information extraction system into a real banking environment reduced cycle times substantially. When compared to the manual workflow, document processing pipeline shortened book-to-book money transfers to 10 minutes (from 29 min.) and electronic fund transfers (EFT) to 17 minutes (from 41 min.) respectively.

DeNERT-KG: Named Entity and Relation Extraction Model Using DQN, Knowledge Graph, and BERT

Along with studies on artificial intelligence technology, research is also being carried out actively in the field of natural language processing to understand and process people’s language, in other words, natural language. For computers to learn on their own, the skill of understanding natural language is very important. There are a wide variety of tasks involved in the field of natural language processing, but we would like to focus on the named entity registration and relation extraction task, which is considered to be the most important in understanding sentences. We propose DeNERT-KG, a model that can extract subject, object, and relationships, to grasp the meaning inherent in a sentence. Based on the BERT language model and Deep Q-Network, the named entity recognition (NER) model for extracting subject and object is established, and a knowledge graph is applied for relation extraction. Using the DeNERT-KG model, it is possible to extract the subject, type of subject, object, type of object, and relationship from a sentence, and verify this model through experiments.

Recognizing Chinese judicial named entity using BiLSTM-CRF

Named entity recognition (NER) plays an essential role in natural language processing systems. Judicial NER is a fundamental component of judicial information retrieval, entity relation extraction, and knowledge map building. However, Chinese judicial NER remains to be more challenging due to the characteristics of Chinese and high accuracy requirements in the judicial filed. Thus, in this paper, we propose a deep learning-based method named BiLSTM-CRF which consists of bi-directional long short-term memory (BiLSTM) and conditional random fields (CRF). For further accuracy promotion, we propose to use Adaptive moment estimation (Adam) for optimization of the model. To validate our method, we perform experiments on judgment documents including commutation, parole and temporary service outside prison, which is acquired from China Judgments Online. Experimental results achieve the accuracy of 0.876, recall of 0.856 and F1 score of 0.855, which suggests the superiority of the proposed BiLSTM-CRF with Adam optimizer.

Medical Information Extraction in the Age of Deep Learning.

Summary Objectives : We survey recent developments in medical Information Extraction (IE) as reported in the literature from the past three years. Our focus is on the fundamental methodological paradigm shift from standard Machine Learning (ML) techniques to Deep Neural Networks (DNNs). We describe applications of this new paradigm concentrating on two basic IE tasks, named entity recognition and relation extraction, for two selected semantic classes—diseases and drugs (or medications)—and relations between them. Methods : For the time period from 2017 to early 2020, we searched for relevant publications from three major scientific communities: medicine and medical informatics, natural language processing, as well as neural networks and artificial intelligence. Results : In the past decade, the field of Natural Language Processing (NLP) has undergone a profound methodological shift from symbolic to distributed representations based on the paradigm of Deep Learning (DL). Meanwhile, this trend is, although with some delay, also reflected in the medical NLP community. In the reporting period, overwhelming experimental evidence has been gathered, as illustrated in this survey for medical IE, that DL-based approaches outperform non-DL ones by often large margins. Still, small-sized and access-limited corpora create intrinsic problems for data-greedy DL as do special linguistic phenomena of medical sublanguages that have to be overcome by adaptive learning strategies. Conclusions : The paradigm shift from (feature-engineered) ML to DNNs changes the fundamental methodological rules of the game for medical NLP. This change is by no means restricted to medical IE but should also deeply influence other areas of medical informatics, either NLP- or non-NLP-based.

A transition-based neural framework for Chinese information extraction.

Chinese information extraction is traditionally performed in the process of word segmentation, entity recognition, relation extraction and event detection. This pipelined approach suffers from two limitations: 1) It is prone to introduce propagated errors from upstream tasks to subsequent applications; 2) Mutual benefits of cross-task dependencies are hard to be introduced in non-overlapping models. To address these two challenges, we propose a novel transition-based model that jointly performs entity recognition, relation extraction and event detection as a single task. In addition, we incorporate subword-level information into character sequence with the use of a hybrid lattice structure, removing the reliance of external word tokenizers. Results on standard ACE benchmarks show the benefits of the proposed joint model and lattice network, which gives the best result in the literature.

A transition-based neural framework for Chinese nested entity and relation recognition

Entity and relation extraction for Chinese texts are typical performed in a pipelined fashion in the sense that by first segmenting sequence into words then recognizing entities and relations subsequently. However, this process often leads to the problem of error propagation and prevents cross-task information integration. To address this issue, we propose a novel transition-based model that performs nested entity recognition and relation extraction jointly without the need of word segmentation beforehand, which is achieved by leveraging the recent advance in using a lattice structure for Chinese sentence encodings. On standard ACE benchmarks, our model gives the best results in the literature. Further analyses show the effectiveness of the proposed architecture in capturing structural outputs.

A Survey Deep Learning Based Relation Extraction

From human understanding of natural language to machine understanding of natural language, NLP has become the mainstream technology in the world.Extracting useful information from massive information, namely information extraction (IE), such as relation extraction (RE), is one of the important semantic processing tasks.With the explosion of web texts and the emergence of new relations, the margin of human knowledge has increased dramatically, and unstructured text processing has become a problem that needs to be overcome, so people have been working on RE for many years. From the earliest pattern matching to the current popular neural network, RE has made significant progress. After reviewing the development history of relation extraction, this paper analyzes and discuss the development of existing neural network models and pre-trained language models from static technology to dynamic technology and reinforcement learning; finally, combined with the latest development of NLP research technology, the future research direction and trend of entity relation extraction are prospected.

Enriching contextualized language model from knowledge graph for biomedical information extraction.

Biomedical information extraction (BioIE) is an important task. The aim is to analyze biomedical texts and extract structured information such as named entities and semantic relations between them. In recent years, pre-trained language models have largely improved the performance of BioIE. However, they neglect to incorporate external structural knowledge, which can provide rich factual information to support the underlying understanding and reasoning for biomedical information extraction. In this paper, we first evaluate current extraction methods, including vanilla neural networks, general language models and pre-trained contextualized language models on biomedical information extraction tasks, including named entity recognition, relation extraction and event extraction. We then propose to enrich a contextualized language model by integrating a large scale of biomedical knowledge graphs (namely, BioKGLM). In order to effectively encode knowledge, we explore a three-stage training procedure and introduce different fusion strategies to facilitate knowledge injection. Experimental results on multiple tasks show that BioKGLM consistently outperforms state-of-the-art extraction models. A further analysis proves that BioKGLM can capture the underlying relations between biomedical knowledge concepts, which are crucial for BioIE.

Boundaries and edges rethinking: An end-to-end neural model for overlapping entity relation extraction

Overlapping entity relation extraction has received extensive research attention in recent years. However, existing methods suffer from the limitation of long-distance dependencies between entities, and fail to extract the relations when the overlapping situation is relatively complex. This issue limits the performance of the task. In this paper, we propose an end-to-end neural model for overlapping relation extraction by treating the task as a quintuple prediction problem. The proposed method first constructs the entity graphs by enumerating possible candidate spans, then models the relational graphs between entities via a graph attention model. Experimental results on five benchmark datasets show that the proposed model achieves the current best performance, outperforming previous methods and baseline systems by a large margin. Further analysis shows that our model can effectively capture the long-distance dependencies between entities in a long sentence.

Automatic Semantic Description Extraction from Social Big Data for Emergency Management.

Emergency events are unexpected and dangerous situations which the authorities must manage and respond to as quickly as possible. The main objectives of emergency management are to provide human safety and security, and Social Big Data (SBD) offers an important information source, created directly from eyewitness reports, to assist with these issues. However, the manual extraction of hidden meaning from SBD is both time-consuming and labor-intensive, which are major drawbacks for a process that needs accurate information to be produced in real-time. The solution is an automatic approach to knowledge discovery, and we propose a semantic description technique based on the use of triple store indexing for named entity recognition and relation extraction. Our technique can discover hidden SBD information more effectively than traditional approaches, and can be used for intelligent emergency management.