Named Entity Recognition Research Articles

An Automatic and End-to-End System for Rare Disease Knowledge Graph Construction Based on Ontology-Enhanced Large Language Models: Development Study.

Rare diseases affect millions worldwide but sometimes face limited research focus individually due to low prevalence. Many rare diseases do not have specific International Classification of Diseases, Ninth Edition (ICD-9) and Tenth Edition (ICD-10), codes and therefore cannot be reliably extracted from granular fields like "Diagnosis" and "Problem List" entries, which complicates tasks that require identification of patients with these conditions, including clinical trial recruitment and research efforts. Recent advancements in large language models (LLMs) have shown promise in automating the extraction of medical information, offering the potential to improve medical research, diagnosis, and management. However, most LLMs lack professional medical knowledge, especially concerning specific rare diseases, and cannot effectively manage rare disease data in its various ontological forms, making it unsuitable for these tasks. Our aim is to create an end-to-end system called automated rare disease mining (AutoRD), which automates the extraction of rare disease-related information from medical text, focusing on entities and their relations to other medical concepts, such as signs and symptoms. AutoRD integrates up-to-date ontologies with other structured knowledge and demonstrates superior performance in rare disease extraction tasks. We conducted various experiments to evaluate AutoRD's performance, aiming to surpass common LLMs and traditional methods. AutoRD is a pipeline system that involves data preprocessing, entity extraction, relation extraction, entity calibration, and knowledge graph construction. We implemented this system using GPT-4 and medical knowledge graphs developed from the open-source Human Phenotype and Orphanet ontologies, using techniques such as chain-of-thought reasoning and prompt engineering. We quantitatively evaluated our system's performance in entity extraction, relation extraction, and knowledge graph construction. The experiment used the well-curated dataset RareDis2023, which contains medical literature focused on rare disease entities and their relations, making it an ideal dataset for training and testing our methodology. On the RareDis2023 dataset, AutoRD achieved an overall entity extraction F1-score of 56.1% and a relation extraction F1-score of 38.6%, marking a 14.4% improvement over the baseline LLM. Notably, the F1-score for rare disease entity extraction reached 83.5%, indicating high precision and recall in identifying rare disease mentions. These results demonstrate the effectiveness of integrating LLMs with medical ontologies in extracting complex rare disease information. AutoRD is an automated end-to-end system for extracting rare disease information from text to build knowledge graphs, addressing critical limitations of existing LLMs by improving identification of these diseases and connecting them to related clinical features. This work underscores the significant potential of LLMs in transforming health care, particularly in the rare disease domain. By leveraging ontology-enhanced LLMs, AutoRD constructs a robust medical knowledge base that incorporates up-to-date rare disease information, facilitating improved identification of patients and resulting in more inclusive research and trial candidacy efforts.

Domain Ontology-Driven Knowledge Graph Generation from Text

A knowledge graph serves as a unified and standardized representation for extracting and representing textual information. In the field of knowledge extraction and representation research, named entity recognition and relation extraction provide effective solutions for knowledge graph generation tasks. However, it is a challenge that lies in extracting domain-specific knowledge from the rich and general textual corpora and generating corresponding domain knowledge graphs to support domain-specific reasoning, question-answering, and decision-making tasks. The hierarchical domain knowledge representation model (i.e. domain ontology) provides a solution for this problem. Therefore, we propose an end-to-end approach based on domain ontology embedding and pre-trained language models for domain knowledge graph generation from text, which incorporates domain node recognition and domain relation extraction phases. We evaluated our domain ontology-driven model on the Wikidata-TekGen dataset and the DBpedia-WebNLG dataset, and the results indicate that our approach based on the pre-trained language models with fewer parameters compared with the baseline models has significantly contributed to the domain knowledge graph generation without prompts.

Low-Resource Chinese Named Entity Recognition via CNN-based Multitask Learning

Named entity recognition (NER) is a fundamental subtask for information extraction that aims to locate and classify named entities in unstructured text into predefined categories. Recently, large-scale language models (LLMs) have achieved SOTA performance on a variety of natural language processing tasks. However, because NER is a sequence labeling task in nature while LLMs is a text-generation model, the performance of LLMs on NER is still significantly below supervised baselines, and NER remains a difficult task. Meanwhile, the word boundary and semantic information of Chinese words are usually quite vague, as words contained in Chinese texts are not separated by spaces. Thus, the NER task still requires supervised learning paradigm and heavily relies on large amounts of labeled data, such as entity type and boundary information. However, the cost of labeling data can be prohibitively large, and the purely supervised approaches usually suffer from poor generalization capability. In this article, we propose a multitask learning-based bidirectional iterated dilated convolution model, BCNN-CWS, for low-resource NER via leveraging word boundary information of Chinese word segmentation (CWS) task. Specifically, to efficiently recognize named entities, an iterated dilated convolutional model with a limited number of layers is implemented. In addition, a bidirectional causal convolution mechanism is presented for contextual information extraction. Results of extensive experiments on public Chinese datasets demonstrate that BCNN-CWS achieves superior performance over state-of-the-art models, and it yields up to about 50% speed improvement over existing methods. It is worth noting that BCNN-CWS can be further improved by combining with a pretrained model. Received: 25 Spetember 2024 | Revised: 4 November 2024 | Accepted: 28 November 2024 Conflicts of Interest The authors declare that they have no conflicts of interest to this work. Data Availability Statement The data that support the findings of this study are openly available in GitLab at https://github.com/jiangfeng13/BCNN-CWS Author Contribution Statement Tao Wu: Conceptualization, Methodology, Writing – original draft, Writing – review & editing, Visualization, Supervision. Xinwen Cao: Resources, Data curation. Feng Jiang: Software, Validation, Formal analysis, Investigation, Writing – original draft. Canyixing Cui: Data curation, Writing -review & editing. Xuehao Li: Resources. Xingping Xian: Supervision, Project administration, Funding acquisition.

Improved self-training-based distant label denoising method for cybersecurity entity extractions.

The task of named entity recognition (NER) plays a crucial role in extracting cybersecurity-related information. Existing approaches for cybersecurity entity extraction predominantly rely on manual labelling data, resulting in labour-intensive processes due to the lack of a cybersecurity-specific corpus. In this paper, we propose an improved self-training-based distant label denoising method for cybersecurity entity extraction. Firstly, we create two domain dictionaries of cybersecurity. Then, an algorithm that combines reverse maximum matching and part-of-speech tagging restrictions is proposed, for generating distant labels for the cybersecurity domain corpus. Lastly, we propose a high-confidence text selection method and an improved self-training algorithm that incorporates a teacher-student model and weight update constraints, for exploring the true labels of low-confidence text using a model trained on high-confidence text, thereby reducing the noise in the distant annotation data. Experimental results demonstrate that the cybersecurity distantly-labelled data we obtained is of high quality. Additionally, the proposed constrained self-training algorithm effectively improves the F1 score of several state-of-the-art NER models on this dataset, yielding a 3.5% improvement for the Vendor class and a 3.35% improvement for the Product class.

Evaluating Chatbot Assistance in Historical Document Analysis

The article explores the potential of PDFGear Copilot, a chatbot-based PDF editing tool, in assisting with the analysis of historical documents. We evaluated the chatbot's performance on a document relating to the Slovenian War of Independence. We included 25 factual and 5 interpretative questions to address its formal characteristics and content details, assess its capacity for in-depth interpretation and contextualized critical analysis, and evaluate the chatbot’s language use and robustness. The chatbot exhibited some ability to answer factual questions, even though its performance varied. It demonstrated proficiency in navigating document structure, named entity recognition, and extracting basic document information. However, performance declined significantly in tasks such as document type identification, content details, and tasks requiring deeper text analysis. For interpretative questions, the chatbot's performance was notably inadequate, failing to link cause-and-effect relationships and provide the depth and nuance required for historical inquiries.

Trustworthy AI: Securing Sensitive Data in Large Language Models

Large language models (LLMs) have transformed Natural Language Processing (NLP) by enabling robust text generation and understanding. However, their deployment in sensitive domains like healthcare, finance, and legal services raises critical concerns about privacy and data security. This paper proposes a comprehensive framework for embedding trust mechanisms into LLMs to dynamically control the disclosure of sensitive information. The framework integrates three core components: User Trust Profiling, Information Sensitivity Detection, and Adaptive Output Control. By leveraging techniques such as Role-Based Access Control (RBAC), Attribute-Based Access Control (ABAC), Named Entity Recognition (NER), contextual analysis, and privacy-preserving methods like differential privacy, the system ensures that sensitive information is disclosed appropriately based on the user’s trust level. By focusing on balancing data utility and privacy, the proposed solution offers a novel approach to securely deploying LLMs in high-risk environments. Future work will focus on testing this framework across various domains to evaluate its effectiveness in managing sensitive data while maintaining system efficiency.

Extracting Design Rationale in Technical Literature to Provide Inspirational Design Stimuli

Abstract Design Rationale (DR) explains why the solution is designed the way it is, and can be used to stimulate creativity and facilitate the development of new solutions in the conceptual design phase. DR was mainly captured by recording the tacit knowledge of designers during the design process, which has hindered its application in conceptual design due to its interference with the design. This paper proposes a method for capturing DR from technical literature, providing an intuitively understandable textual stimulus for design ideation. A Textual DR Ontology (TDRO), which includes literature, artifact, issue, intention, argument and other entities along with their relationships, is used as a metamodel to construct the DR Knowledge Graph (DRKG). The DR Vector Space (DRVS) model and the DRVS-based method are used for the joint extraction of entities and relations. Sentences and terms extracted from the technical literature are then organized into a DRKG. A prototype Design Rationale Management System (DRMS) was developed based on the methodology. Finally, we carried out experiments to construct the DRKG and apply it to the conceptual design of a police Unmanned Aerial Vehicle (UAV) for night patrols using patents and journal articles, and the results verified the feasibility of the method.

Single-Stage Entity–Relation Joint Extraction of Pesticide Registration Information Based on HT-BES Multi-Dimensional Labeling Strategy

Pesticide registration information is an essential part of the pesticide knowledge base. However, the large amount of unstructured text data that it contains pose significant challenges for knowledge storage, retrieval, and utilization. To address the characteristics of pesticide registration text such as high information density, complex logical structures, large spans between entities, and heterogeneous entity lengths, as well as to overcome the challenges faced when using traditional joint extraction methods, including triplet overlap, exposure bias, and redundant computation, we propose a single-stage entity–relation joint extraction model based on HT-BES multi-dimensional labeling (MD-SERel). First, in the encoding layer, to address the complex structural characteristics of pesticide registration texts, we employ RoBERTa combined with a multi-head self-attention mechanism to capture the deep semantic features of the text. Simultaneously, syntactic features are extracted using a syntactic dependency tree and graph neural networks to enhance the model’s understanding of text structure. Subsequently, we integrate semantic and syntactic features, enriching the character vector representations and thus improving the model’s ability to represent complex textual data. Secondly, in the multi-dimensional labeling framework layer, we use HT-BES multi-dimensional labeling, where the model assigns multiple labels to each character. These labels include entity boundaries, positions, and head–tail entity association information, which naturally resolves overlapping triplets. Through utilizing a parallel scoring function and fine-grained classification components, the joint extraction of entities and relations is transformed into a multi-label sequence labeling task based on relation dimensions. This process does not involve interdependent steps, thus enabling single-stage parallel labeling, preventing exposure bias and reducing computational redundancy. Finally, in the decoding layer, entity–relation triplets are decoded based on the predicted labels from the fine-grained classification. The experimental results demonstrate that the MD-SERel model performs well on both the Pesticide Registration Dataset (PRD) and the general DuIE dataset. On the PRD, compared to the optimal baseline model, the training time is 1.2 times faster, the inference time is 1.2 times faster, and the F1 score is improved by 1.5%, demonstrating its knowledge extraction capabilities in pesticide registration documents. On the DuIE dataset, the MD-SERel model also achieved better results compared to the baseline, demonstrating its strong generalization ability. These findings will provide technical support for the construction of pesticide knowledge bases.

Deep learning models can predict violence and threats against healthcare providers using clinical notes

Violence, verbal abuse, threats, and sexual harassment of healthcare providers by patients is a major challenge for healthcare organizations around the world, contributing to staff turnover, distress, absenteeism, reduced job satisfaction, and worsening mental and physical health. To enable interventions prior to possible violent episodes, we trained two deep learning models to predict violence against healthcare workers 3 days prior to violent events for case and control patients. The first model is a document classification model using clinical notes, and the second is a baseline regression model using largely structured data. Our document classification model achieved an F1 score of 0.75 while our model using structured data achieved an F1 of 0.72, both exceeding the predictive performance of a psychiatry team who reviewed the same documents (0.5 F1). To aid in the explainability and understanding of risk factors for violent events, we additionally trained a named entity recognition classifier on annotations of the same corpus, which achieved an overall F1 of 0.7. This study demonstrates the first deep learning model capable of predicting violent events within healthcare settings using clinical notes, surpassing the first published baseline of human experts. We anticipate our methods can be generalized and extended to enable intervention at other hospital systems.

Automated derivation of diagnostic criteria for lung cancer using natural language processing on electronic health records: a pilot study

BackgroundThe digitisation of healthcare records has generated vast amounts of unstructured data, presenting opportunities for improvements in disease diagnosis when clinical coding falls short, such as in the recording of patient symptoms. This study presents an approach using natural language processing to extract clinical concepts from free-text which are used to automatically form diagnostic criteria for lung cancer from unstructured secondary-care data.MethodsPatients aged 40 and above who underwent a chest x-ray (CXR) between 2016 and 2022 were included. ICD-10 and unstructured data were pulled from their electronic health records (EHRs) over the preceding 12 months to the CXR. The unstructured data were processed using named entity recognition to extract symptoms, which were mapped to SNOMED-CT codes. Subsumption of features up the SNOMED-CT hierarchy was used to mitigate against sparse features and a frequency-based criteria, combined with univariate logarithmic probabilities, was applied to select candidate features to take forward to the model development phase. A genetic algorithm was employed to identify the most discriminating features to form the diagnostic criteria.Results75002 patients were included, with 1012 lung cancer diagnoses made within 12 months of the CXR. The best-performing model achieved an AUROC of 0.72. Results showed that an existing ‘disorder of the lung’, such as pneumonia, and a ‘cough’ increased the probability of a lung cancer diagnosis. ‘Anomalies of great vessel’, ‘disorder of the retroperitoneal compartment’ and ‘context-dependent findings’, such as pain, statistically reduced the risk of lung cancer, making other diagnoses more likely. The performance of the developed model was compared to the existing cancer risk scores, demonstrating superior performance.ConclusionsThe proposed methods demonstrated success in leveraging unstructured secondary-care data to derive diagnostic criteria for lung cancer, outperforming existing risk tools. These advancements show potential for enhancing patient care and results. However, it is essential to tackle specific limitations by integrating primary care data to ensure a more thorough and unbiased development of diagnostic criteria. Moreover, the study highlights the importance of contextualising SNOMED-CT concepts into meaningful terminology that resonates with clinicians, facilitating a clearer and more tangible understanding of the criteria applied.

Exploring Named Entity Recognition via MacBERT-BiGRU and Global Pointer with Self-Attention

Named Entity Recognition (NER) is a fundamental task in natural language processing that aims to identify and categorize named entities within unstructured text. In recent years, with the development of deep learning techniques, pre-trained language models have been widely used in NER tasks. However, these models still face limitations in terms of their scalability and adaptability, especially when dealing with complex linguistic phenomena such as nested entities and long-range dependencies. To address these challenges, we propose the MacBERT-BiGRU-Self Attention-Global Pointer (MB-GAP) model, which integrates MacBERT for deep semantic understanding, BiGRU for rich contextual information, self-attention for focusing on relevant parts of the input, and a global pointer mechanism for precise entity boundary detection. By optimizing the number of attention heads and global pointer heads, our model achieves an effective balance between complexity and performance. Extensive experiments on benchmark datasets, including ResumeNER, CLUENER2020, and SCHOLAT-School, demonstrate significant improvements over baseline models.

Voting Strategies for Arabic Named Entity Recognition using Annotation Schemes

Named Entity Recognition (NER) seeks to identify and classify NEs into predefined categories and is an important subtask in information extraction. Many annotation schemes have been proposed to assign suitable labels for multiword NEs within a given text. This study proposes a method to combine the results of different annotation schemes (IOB, IOE, IOBE, IOBS, IOES, and IOBES) for Arabic NER (ANER). Three voting strategies are explored, namely, majority voting, weighted voting, and weighted voting-based Particle Swarm Optimization (PSO), applied to Conditional Random Fields (CRF) classifiers, each corresponding to a certain annotation scheme. The experimental results showed that majority voting can be considered an effective combination strategy to enhance the performance of ANER systems.

MPE[formula omitted]: Learning meta-prompt with entity-enhanced semantics for few-shot named entity recognition

Recently, prompt-tuning has been proven to be surprisingly effective on few-shot tasks. Intuitively, some studies explore Few-shot Named Entity Recognition (NER) based on prompt-tuning. However, how to properly initialize and effectively learn the prompt under limited training conditions still remains significantly challenging for few-shot NER. To meet these challenges, we propose a novel Meta-Prompt with Entity-Enhanced semantics for Few-shot NER, MPE3 for brevity. Specifically, we first explore the importance of the named entities’ semantics in the few-shot NER task. And we propose to construct prompts with entity-enhanced semantics which contain much useful prior knowledge for identifying named entities. Furthermore, to address the issue of inadequate training more substantially, we aim to train a meta-prompt that can be more effective and adaptive for few-shot NER scenarios. To achieve this, we divide the training data into many source-domain agnostic meta-tasks tailored to the characteristics of the NER problem for training. And we specially design a prompt meta-learner for training these meta-tasks. This training strategy succeeds in guiding prompts to optimize in a better direction for few-shot scenarios by the learned meta-knowledge from each meta-task. We conduct extensive experiments on three NER datasets under two different few-shot settings. Our method outperforms the current state-of-the-art model by 5.60%∼13.34% and 2.41%∼7.34% on average in the two different few-shot settings respectively, which validates the effectiveness and superiority of our model.

ALDANER: Active Learning based Data Augmentation for Named Entity Recognition

Training Named Entity Recognition (NER) models typically necessitates the use of extensively annotated datasets. This requirement presents a significant challenge due to the labor-intensive and costly nature of manual annotation, especially in specialized domains such as medicine and finance. To address data scarcity, two strategies have emerged as effective: (1) Active Learning (AL), which autonomously identifies samples that would most enhance model performance if annotated, and (2) data augmentation, which automatically generates new samples. However, while AL reduces human effort, it does not eliminate it entirely, and data augmentation often leads to incomplete and noisy annotations, presenting new hurdles in NER model training. In this study, we integrate AL principles into a data augmentation framework, named Active Learning-based Data Augmentation for NER (ALDANER), to prioritize the selection of informative samples from an augmented pool and mitigate the impact of noisy annotations. Our experiments across various benchmark datasets and few-shot scenarios demonstrate that our approach surpasses several data augmentation baselines, offering insights into promising avenues for future research.

Named Entity Recognition and Relationship Extraction for Biomedical Text: A comprehensive survey, recent advancements, and future research directions

Named Entity Recognition and Relationship Extraction for Biomedical Text: A comprehensive survey, recent advancements, and future research directions

Local Metric NER: A new paradigm for named entity recognition from a multi-label perspective

As the field of Nested Named Entity Recognition (NNER) advances, it is marked by a growing complexity due to the increasing number of multi-label entity instances. How to more effectively identify multi-label entities and explore the correlation between labels is the focus of our work. Unlike previous models that are modeled in single-label multi-classification problems, we propose a novel multi-label local metric NER model to rethink Nested Entity Recognition from a multi-label perspective. Simultaneously, to address the significant sample imbalance problem commonly encountered in multi-label scenarios, we introduce a parts-of-speech-based strategy that significantly improves the model’s performance on imbalanced datasets. Experiments on nested, multi-label, and flat datasets verify the generalization and superiority of our model, with results surpassing the existing state-of-the-art (SOTA) on several multi-label and flat benchmarks. After a series of experimental analyses, we highlight the persistent challenges in the multi-label NER. We are hopeful that the insights derived from our work will not only provide new perspectives on the nested NER landscape but also contribute to the ongoing momentum necessary for advancing research in the field of multi-label NER.

Cross-domain Constituency Parsing by Leveraging Heterogeneous Data

Knowledge transfer is investigated in various natural language processing tasks except cross-domain constituency parsing. In this paper, we leverage heterogeneous data to transfer cross-domain and cross-task knowledge to constituency parsing. Concretely, we first select language modeling, named entity recognition, CCG supertagging and dependency parsing as auxiliary tasks and collect the corpora of these tasks covering various domains as cross-domain and cross-task heterogeneous data. Second, we exploit three types of prefixes: shared, task and domain prefix, to merge cross-domain and cross-task data and decompose the general, task and domain representation in the pretrained language model. Third, we convert the data formats of multi-source heterogeneous datasets and loss objectives of the auxiliary tasks into a consistent formalization closer to constituency parsing. Finally, we jointly train the model to transfer task and domain knowledge to cross-domain constituency parsing. We verify the effectiveness of our proposed model on five target domains of MCTB. Experimental results show that our knowledge transfer model outperforms various baseline models, including conventional chart-based and transition-based parsers and the current large-scale language model for zero-shot and few-shot settings.

Exploring the effectiveness of instruction tuning in biomedical language processing

Large Language Models (LLMs), particularly those similar to ChatGPT, have significantly influenced the field of Natural Language Processing (NLP). While these models excel in general language tasks, their performance in domain-specific downstream tasks such as biomedical and clinical Named Entity Recognition (NER), Relation Extraction (RE), and Medical Natural Language Inference (NLI) is still evolving. In this context, our study investigates the potential of instruction tuning for biomedical language processing, applying this technique to two general LLMs of substantial scale. We present a comprehensive, instruction-based model trained on a dataset that consists of approximately 200,000 instruction-focused samples. This dataset represents a carefully curated compilation of existing data, meticulously adapted and reformatted to align with the specific requirements of our instruction-based tasks. This initiative represents an important step in utilising such models to achieve results on par with specialised encoder-only models like BioBERT and BioClinicalBERT for various classical biomedical NLP tasks. Our work includes an analysis of the dataset’s composition and its impact on model performance, providing insights into the intricacies of instruction tuning. By sharing our codes, models, and the distinctively assembled instruction-based dataset, we seek to encourage ongoing research and development in this area.22Our code repository is available at https://github.com/nlpie-research/BioInstTune-LLM.

Improving Few-Shot Named Entity Recognition with Causal Interventions

Improving Few-Shot Named Entity Recognition with Causal Interventions

Evaluation of the Usage of Graph Database in a Nutrition App for Determining Potential Nutrient Inhibition

Abstract In recent years, the popularity of veganism has surged, prompting a growing interest in its health implications. While vegan diets offer numerous benefits, they also pose challenges due to potential deficiencies in essential nutrients and the presence of anti-nutrients in plant-based foods. This paper addresses the need for a comprehensive understanding of nutrient interactions, particularly those involving antinutrients, by creating a Knowledge Graph. Utilizing Natural Language Processing techniques, essential nutritional entities were extracted from unstructured texts, and a custom Named Entity Recognition model was implemented. These data were then used to construct a Knowledge Graph using Neo4j, a leading graph database. Despite the potential of the developed Knowledge Graph to visualize nutrient interactions, limitations persist due to the scarcity of available data on antinutrient interactions. Nonetheless, the Knowledge Graph provides a promising avenue for exploring and understanding the complexities of nutrient interactions, facilitating future research and informing dietary choices.