Entity Relation Extraction Research Articles

探索中文文本中实体关系智能提取：一种基于数据与模型协同优化的新方法

探索中文文本中实体关系智能提取：一种基于数据与模型协同优化的新方法

CPBA-CLIM: An entity-relation extraction model for ontology-based knowledge graph construction in hazardous chemical incident management.

In recent years, hazardous chemical incidents have occurred frequently, resulting in significant human casualties, property damage, and environmental pollution due to human or natural factors. Accurately mining the lessons learned from accumulating incident reports and constructing the knowledge graph for hazardous chemical incident management can assist managers in identifying patterns and analyzing common attributes, thereby preventing the recurrence of similar incidents. This article addresses the challenges of dispersed textual information, specialized vocabulary, and data formats in hazardous chemical incidents. We propose a novel entity-relation extraction model called CPBA-CLIM (content-position-based attention-cross-label intersect matching) to provide an accurate data foundation for constructing the hazardous chemical incident knowledge graph. The content-position-based attention module, based on content-position attention, incorporates contextual semantic information into the combined encoding of bidirectional encoder representations from the transformer's content and position to obtain dynamic word vectors that align with the thematic context of the text. Additionally, the cross-label intersect matching strategy evaluates the rationality of entity-relation interactions in sets containing potential overlaps, reducing the impact of entity-relation overlap on triplet extraction accuracy. Comparative experimental results on public datasets demonstrate the model's outstanding performance in overlapping triplets. Qualitative experiments on a self-constructed dataset integrate our model with ontology construction techniques, successfully establishing a knowledge graph for managing hazardous chemical incidents. This research effectively enhances the degree of automation and efficiency in knowledge graph construction, thus offering support and decision-making foundations for hazardous chemical safety management.

Fault Knowledge Graph Construction and Platform Development for Aircraft PHM.

To tackle the problems of over-reliance on traditional experience, poor troubleshooting robustness, and slow response by maintenance personnel to changes in faults in the current aircraft health management field, this paper proposes the use of a knowledge graph. The knowledge graph represents troubleshooting in a new way. The aim of the knowledge graph is to improve the correlation between fault data by representing experience. The data source for this study consists of the flight control system manual and typical fault cases of a specific aircraft type. A knowledge graph construction approach is proposed to construct a fault knowledge graph for aircraft health management. Firstly, the data are classified using the ERNIE model-based method. Then, a joint entity relationship extraction model based on ERNIE-BiLSTM-CRF-TreeBiLSTM is introduced to improve entity relationship extraction accuracy and reduce the semantic complexity of the text from a linguistic perspective. Additionally, a knowledge graph platform for aircraft health management is developed. The platform includes modules for text classification, knowledge extraction, knowledge auditing, a Q&A system, and graph visualization. These modules improve the management of aircraft health data and provide a foundation for rapid knowledge graph construction and knowledge graph-based fault diagnosis.

LMKG: A large-scale and multi-source medical knowledge graph for intelligent medicine applications

Medical Knowledge Graph (KG) has shown great potential in various healthcare scenarios, such as drug recommendation and clinical decision support system. The factors that determine the role of a medical KG in practical applications are the scale, coverage, and quality of the medical knowledge it can provide. Most existing medical KGs are extracted from a single or a few information sources. However, medical knowledge extracted from insufficient information sources is usually highly incomplete or even biased, which results in a lack of data completeness and may lessen their effectiveness in real-world scenarios. Besides, the coverage of entity and relation types is inadequate in most previous works, which also might restrict their potential usage in future applications. In this paper, we build a unified system that can extract and manage medical knowledge from heterogeneous information sources. We first employ named entity recognition and relation extraction methods to extract knowledge triplets from medical texts. Then we propose a hierarchical entity alignment framework for further knowledge refinement. Based on our system, we construct a large-scale, high-quality, multi-source, and multi-lingual medical KG named LMKG, which includes 13 entity types and 17 relation types, and contains 403,784 entity and 1,225,097 relation instances. We conduct extensive experiments to evaluate the quality of LMKG. Experimental results show that LMKG can effectively enhance the performance of both upstream and downstream intelligent medicine applications. We have publicly released the KG resources and corresponding management service interface to facilitate research and applications in the medical field.

APIE: An information extraction module designed based on the pipeline method

Information extraction (IE) aims to discover and extract valuable information from unstructured text. This problem can be decomposed into two subtasks: named entity recognition (NER) and relation extraction (RE). Although the IE problem has been studied for years, most work efforts focused on jointly modeling these two subtasks, either by casting them into a structured prediction framework or by performing multitask learning through shared representations. However, since the contextual representations of entity and relation models inherently capture different feature information, sharing a single encoder to capture the information required by both subtasks in the same space would harm the accuracy of the model. Recent research (Zhong and Chen, 2020) has also proved that using two separate encoders for NER and RE tasks respectively through pipeline method are effective, with the model surpassing all previous joint models in accuracy. Thus, in this paper, we design An Pipeline method Information Extraction module called APIE, APIE combines the advantages of both pipeline methods and joint methods, demonstrating higher accuracy and powerful reasoning abilities. Specifically, we design a multi-level feature NER model based on attention mechanism and a document-level RE model based on local context pooling. To demonstrate the effectiveness of our proposed approach, we conducted tests on multiple datasets. Extensive experimental results have shown that our proposed model outperforms state-of-the-art methods and improves both accuracy and reasoning abilities.

Joint Entity Relation Extraction Based on LSTM via Attention Mechanism

Joint Entity Relation Extraction Based on LSTM via Attention Mechanism

Extracting Clinical Information From Japanese Radiology Reports Using a 2-Stage Deep Learning Approach: Algorithm Development and Validation.

Radiology reports are usually written in a free-text format, which makes it challenging to reuse the reports. For secondary use, we developed a 2-stage deep learning system for extracting clinical information and converting it into a structured format. Our system mainly consists of 2 deep learning modules: entity extraction and relation extraction. For each module, state-of-the-art deep learning models were applied. We trained and evaluated the models using 1040 in-house Japanese computed tomography (CT) reports annotated by medical experts. We also evaluated the performance of the entire pipeline of our system. In addition, the ratio of annotated entities in the reports was measured to validate the coverage of the clinical information with our information model. The microaveraged F1-scores of our best-performing model for entity extraction and relation extraction were 96.1% and 97.4%, respectively. The microaveraged F1-score of the 2-stage system, which is a measure of the performance of the entire pipeline of our system, was 91.9%. Our system showed encouraging results for the conversion of free-text radiology reports into a structured format. The coverage of clinical information in the reports was 96.2% (6595/6853). Our 2-stage deep system can extract clinical information from chest and abdomen CT reports accurately and comprehensively.

Unearthing historical insights: semantic organization and application of historical newspapers from a fine-grained knowledge element perspective

PurposeThis paper aims to amplify the retrieval and utilization of historical newspapers through the application of semantic organization, all from the vantage point of a fine-grained knowledge element perspective. This endeavor seeks to unlock the latent value embedded within newspaper contents while simultaneously furnishing invaluable guidance within methodological paradigms for research in the humanities domain.Design/methodology/approachAccording to the semantic organization process and knowledge element concept, this study proposes a holistic framework, including four pivotal stages: knowledge element description, extraction, association and application. Initially, a semantic description model dedicated to knowledge elements is devised. Subsequently, harnessing the advanced deep learning techniques, the study delves into the realm of entity recognition and relationship extraction. These techniques are instrumental in identifying entities within the historical newspaper contents and capturing the interdependencies that exist among them. Finally, an online platform based on Flask is developed to enable the recognition of entities and relationships within historical newspapers.FindingsThis article utilized the Shengjing Times·Changchun Compilation as the datasets for describing, extracting, associating and applying newspapers contents. Regarding knowledge element extraction, the BERT + BS consistently outperforms Bi-LSTM, CRF++ and even BERT in terms of Recall and F1 scores, making it a favorable choice for entity recognition in this context. Particularly noteworthy is the Bi-LSTM-Pro model, which stands out with the highest scores across all metrics, notably achieving an exceptional F1 score in knowledge element relationship recognition.Originality/valueHistorical newspapers transcend their status as mere artifacts, evolving into invaluable reservoirs safeguarding the societal and historical memory. Through semantic organization from a fine-grained knowledge element perspective, it can facilitate semantic retrieval, semantic association, information visualization and knowledge discovery services for historical newspapers. In practice, it can empower researchers to unearth profound insights within the historical and cultural context, broadening the landscape of digital humanities research and practical applications.

Contextual Reinforcement, Entity Delimitation and Generative Data Augmentation for Entity Recognition and Relation Extraction in Official Documents

Transformer architectures have become the main component of various state-of-the-art methods for natural language processing tasks, such as Named Entity Recognition and Relation Extraction (NER+RE). As these architectures rely on semantic (contextual) aspects of word sequences, they may fail to accurately identify and delimit entity spans when there is little semantic context surrounding the named entities. This is the case of entities composed only by digits and punctuation, such as IDs and phone numbers, as well as long composed names. In this article, we propose new techniques for contextual reinforcement and entity delimitation based on pre- and post-processing techniques to provide a richer semantic context, improving SpERT, a state-of-the-art Span-based Entity and Relation Transformer. To provide further context to the training process of NER+RE, we propose a data augmentation technique based on Generative Pretrained Transformers (GPT). We evaluate our strategies using real data from public administration documents (official gazettes and biddings) and court lawsuits. Our results show that our pre- and post-processing strategies, when used co-jointly, allows significant improvements on NER+ER effectiveness, while we also show the benefits of using GPT for training data augmentation.

Valuable Knowledge Mining: Deep Analysis of Heart Disease and Psychological Causes Based on Large-Scale Medical Data

The task of accurately identifying medical entities and extracting entity relationships from large-scale medical text data has become a hot topic in recent years, aiming to mine potential rules and knowledge. How to conduct in-depth context analysis from biomedical texts, such as medical procedures, diseases, therapeutic drugs, and disease characteristics, and identify valuable knowledge in the medical field is our main research content. Through the process of knowledge mining, a deeper understanding of the complex relationships between various factors in diseases can be gained, which holds significant guiding implications for clinical research. An approach based on context semantic analysis is proposed to realize medical entity recognition and entity relationship extraction. In addition, we build a medical knowledge base related to coronary heart disease and combine the NCBI disease dataset and the medical lexicon dataset extracted from the text as the test data of the experiment. Experimental results show that this model can effectively identify entities in medical text data; the WBC model achieved an F1 score of 89.2% in the experiment, while the CSR model achieved an F1 score of 83.4%, and the result is better than other methods.

Emergency entity relationship extraction for water diversion project based on pre-trained model and multi-featured graph convolutional network.

Using information technology to extract emergency decision-making knowledge from emergency plan documents is an essential means to enhance the efficiency and capacity of emergency management. To address the problems of numerous terminologies and complex relationships faced by emergency knowledge extraction of water diversion project, a multi-feature graph convolutional network (PTM-MFGCN) based on pre-trained model is proposed. Initially, through the utilization of random masking of domain-specific terminologies during pre-training, the model's comprehension of the meaning and application of such terminologies within specific fields is enhanced, thereby augmenting the network's proficiency in extracting professional terminologies. Furthermore, by introducing a multi-feature adjacency matrix to capture a broader range of neighboring node information, thereby enhancing the network's ability to handle complex relationships. Lastly, we utilize the PTM-MFGCN to achieve the extraction of emergency entity relationships in water diversion project, thus constructing a knowledge graph for water diversion emergency management. The experimental results demonstrate that PTM-MFGCN exhibits improvements of 2.84% in accuracy, 4.87% in recall, and 5.18% in F1 score, compared to the baseline model. Relevant studies can effectively enhance the efficiency and capability of emergency management, mitigating the impact of unforeseen events on engineering safety.

A Combined Semantic Dependency and Lexical Embedding RoBERTa Model for Grid Field Relational Extraction

Relationship extraction is a crucial step in the construction of a knowledge graph. In this research, the grid field entity relationship extraction was performed via a labeling approach that used span representation. The subject entity and object entity were used as training instances to bolster the linkage between them. The embedding layer of the RoBERTa pre-training model included word embedding, position embedding, and paragraph embedding information. In addition, semantic dependency was introduced to establish an effective linkage between different entities. To facilitate the effective linkage, an additional lexically labeled embedment was introduced to empower the model to acquire more profound semantic insights. After obtaining the embedding layer, the RoBERTa model was used for multi-task learning of entities and relations. The multi-task information was then fused using the parameter hard sharing mechanism. Finally, after the layer was fully connected, the predicted entity relations were obtained. The approach was tested on a grid field dataset created for this study. The obtained results demonstrated that the proposed model has high performance.

Unleashing the power of context: Contextual association network with cross-task attention for joint relational extraction

Extraction of entities and their relationships remains a challenge in information extraction tasks such as opinion recognition and event detection. Research efforts has been dedicated to training named entity recognition (NER) models and relation extraction (RE) models simultaneously, in order to leverage the semantic associations among such sub-tasks to improve their performance. However, how to reasonably capture the interdependence between these two sub-tasks is still not clear. Well trained joint extraction models often do not generalize well to examples from wide sources due to that existing strategies may overly rely on shallow heuristics via entity mentions and fail to make enough use of contextual associations among such sub-tasks. With this in mind, we propose a novel contextual association network for joint relational extraction (called CARE) with cross-task attention mechanism, which makes use of the correlated contextual information produced by the NER and RE sub-tasks explicitly, and achieves significantly better performance comparing with the state-of-the-art models. We use experimental results on four publicly available datasets to test the efficacy of our proposed model. Experimental results demonstrate that the proposed mechanism can use the semantic information produced by NER and RE sub-tasks to boost one another in a complementary way.

A multiple head selection joint entity-relation extraction model

In the entity extraction task, there are some complex extraction problems, such as nested entity, entity boundary recognition, context ambiguity, and multi-instance entity recognition. Entity nesting is an important challenge in relational extraction. The main reason of entity nesting problem is that the boundary information between entities is not clear. In order to solve the entity nesting problem at the fragment level, while preserving the relationship between fragments with the same characteristics and improving efficiency, we proposed a brand new fragment annotation method. On the basis of traditional fragment annotation method, combined with pointer annotation method, we designed an annotation method of "ergodic enumeration + group mapping". On the basis of this method, an entity extraction model is designed: Span-Extraction Based Entity Extraction Model (LMA). Our model underwent a series of validations in the English data sets New York Times(NYT) and WEBNLG, showing significant improvements over the baseline model F1. It can effectively alleviate the above problems.

Bias detection and knowledge graph comparison analysis for medical record datasets

Abstract In this paper, firstly, we study the reinforcement learning algorithm, which is a related technology of knowledge graph, and use reinforcement learning to structure the application framework of health care knowledge graph and construct the domain dictionary. And based on the BitLab21 dataset, entity relationship annotation is performed on the named entity recognition result set to realize entity relationship extraction of clinical electronic medical records. Then, the HacRED dataset is used to evaluate the deep learning model, and comparison tests and ablation experiments are conducted to verify the effectiveness of reinforcement learning for constructing knowledge graphs, respectively. Finally, a deviation monitoring method and a data feature extraction method for the case dataset are proposed, and a comparative study of different feature extraction methods is conducted to illustrate the advantages of feature fusion in distinguishing abnormal deviation patterns. All three features have relatively good recognition accuracy for normal patterns, which can reach 100%. The error curves perform well, and their training and testing errors converge very quickly to below 0.01 with less than 5 iterations and achieve nearly 100% recognition accuracy. Reinforcement learning-based knowledge mapping of case datasets and feature fusion-based data machine bias detection can improve the efficiency of medical detection.

CDTier: A Chinese Dataset of Threat Intelligence Entity Relationships

Cyber Threat Intelligence (CTI), which is knowledge of cyberspace threats gathered from security data, is critical in defending against cyberattacks.However, there is no open-source CTI dataset for security researchers to effectively apply enormous CTI information for security analysis in the field of threat intelligence, particularly in the field of Chinese threat intelligence. As a result, for network security research and development, this paper constructed a Chinese CTI entity relationship dataset–CDTier, which includes: 1) A threat entity extraction dataset composed of 100 CTI reports, 3744 threat sentences and 4259 threat knowledge objects; 2) A dataset for entity relation extraction including 100 CTI reports, 2598 threat sentences and 2562 knowledge object relations. CDTier is, as far as we know, the first CTI dataset. On the CDTier, we trained 4 models for threat entity extraction and relation extraction using well-established and widely used deep learning methods in the NLP. The results showed that the model trained on CDTier extracts knowledge objects and their relationships described in threat intelligence more accurately. This significantly minimizes threat intelligence analysts' work while assessing threat intelligence. The CDTier may be found at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/MuYu-z/CDTier</uri> .

A contrastive learning framework for safety information extraction in construction

Typically named entity recognition (NER) and relation extraction (RE) from safety documentation (e.g., accident reports) adopt a pipeline processing approach whereby tasks are split into two sub-tasks. As a result, error propagation occurs between components, and useful information from one task may go unexploited by the other. Additionally, training sets to perform NER and RE from safety documentation are often limited and context-specific. Thus, our research addresses the following question: How can we accurately identify entities and extract relations from safety documentation using limited training sets? This paper utilizes ‘contrastive learning’ to tackle our research question. It proposes a contrastive learning-based cascade binary tagging framework (CasRel) to automatically and synchronously extract entities and relations from safety documents. A five-fold cross-validation process is used to validate the effectiveness and feasibility of our approach. Results from the validation process achieve an average precision of 77.8%, recall of 58.7%, and F1-score of 66.9%, outperforming CasRel with an increase of about 10% in precision, 5% in recall, and 7% in F1-score. Thus, our approach can accurately recognize entities and extract relations from safety documentation. The contributions of our study are twofold: (1) an improved unified model is developed to recognize and extract the entity and relation from safety documents to reduce error propagation and improve its accuracy; and (2) the concept of ‘contrastive learning’ is introduced in the design of the joint entity and relation extraction model with limited training sets.

Transformer-based approach to variable typing

The upsurge of multifarious endeavors across scientific fields propelled Big Data in the scientific domain. Despite the advancements in management systems, researchers find that mathematical knowledge remains one of the most challenging to manage due to the latter's inherent heterogeneity. One novel recourse being explored is variable typing where current works remain preliminary and, thus, provide a wide room for contribution. In this study, a primordial attempt to implement the end-to-end Entity Recognition (ER) and Relation Extraction (RE) approach to variable typing was made using the BERT (Bidirectional Encoder Representations from Transformers) model. A micro-dataset was developed for this process. According to our findings, the ER model and RE model, respectively, have Precision of 0.8142 and 0.4919, Recall of 0.7816 and 0.6030, and F1-Scores of 0.7975 and 0.5418. Despite the limited dataset, the models performed at par with values in the literature. This work also discusses the factors affecting this BERT-based approach, giving rise to suggestions for future implementations.

Tell me your position: Distantly supervised biomedical entity relation extraction using entity position marker

A significant amount of textual data has been produced in the biomedical area recently as a result of the advancement of biomedical technologies. Large-scale biomedical data can be automatically obtained with the help of distant supervision. However, the noisy data brought by distant supervision methods makes relation extraction tasks more difficult. Previous work has focused more on how to restore mislabeled relationships, but little attention has been paid to the importance of labeled entity locations for relationship extraction tasks. In this paper, we present a “four-stage” model based on BioBERT and Multi-Instance Learning by using entity position markers. Firstly, the sentence is marked with position. Secondly, BioBERT, a biomedical pre-trained language model, is used in the final sentence feature vector representation not only with the global position marker but also with the start and end marker of both the head and tail entity. Thirdly, the aggregation of sentence vectors in the bag is used as the vector feature of the bag by three aggregation methods, and the performance of different sentence feature vectors combined with different bag encoding methods is discussed. At last, relation classification is performed at the bag level. According to experimental results, the presented model significantly outperforms all baseline models and contributes to noise reduction. In addition, different bag encoding methods need to match corresponding sentence encoding representation to achieve the best performance.

Deep purified feature mining model for joint named entity recognition and relation extraction

Table filling based joint named entity recognition and relation extraction task aims to share representation of subtasks in a table to extract structured knowledge. However, most of existing studies need additional labels and dedicated deep neural networks to learn shared representation, imposing heavy burdens to decoders. More seriously, almost all these models suffer from feature confusion problem, failing to capture purified task-specific features from shared representation to perform subtasks. To address these challenging problems, in this paper we propose a novel and effective Deep puRified fEAture Mining (DREAM) model for joint named entity recognition and relation extraction task, which can automatically capture purified task-specific features to improve the classification performance of subtasks. Specifically, unlike introducing additional labels or dedicated network architectures, we design a new lightweight shared representation learning (LSRL) module by the plainest labels of joint task and thus encodes context by the hybrid convolutional neural networks. Afterwards, a task-aware information bottleneck (TIB) module is proposed to explore the relation between the mutual information of the joint distribution of each subtask and its task-specific features. With the above two modules well obtain shared representation and purified task-specific features, the satisfactory classification results of both subtasks can be guaranteed. Experiment results show that the proposed model is highly effective, obtaining the promising results on three different benchmarks: CoNNL04 (general text), ADE (biomedical text) and SciERC (scientific text). For example, DREAM respectively achieves F1-scores of 78.18%, 80.28% and 44.60% in performing the relation extraction subtask on the CoNNL04, ADE and SciERC datasets. The promising performance indicates that the proposed model can be applied to many practical applications such as biomedical information extraction. The source code is publicly available at https://github.com/SWT-AITeam/DREAM.