Named Entity Recognition Model Research Articles

Vocabulary Matters: An Annotation Pipeline and Four Deep Learning Algorithms for Enzyme Named Entity Recognition.

Enzymes are indispensable in many biological processes, and with biomedical literature growing exponentially, effective literature review becomes increasingly challenging. Natural language processing methods offer solutions to streamline this process. This study aims to develop an annotated enzyme corpus for training and evaluating enzyme named entity recognition (NER) models. A novel pipeline, combining dictionary matching and rule-based keyword searching, automatically annotated enzyme entities in >4800 full-text publications. Four deep learning NER models were created with different vocabularies (BioBERT/SciBERT) and architectures (BiLSTM/transformer) and evaluated on 526 manually annotated full-text publications. The annotation pipeline achieved an F1-score of 0.86 (precision = 1.00, recall = 0.76), surpassed by fine-tuned transformers for F1-score (BioBERT: 0.89, SciBERT: 0.88) and recall (0.86) with BiLSTM models having higher precision (0.94) than transformers (0.92). The annotation pipeline runs in seconds on standard laptops with almost perfect precision, but was outperformed by fine-tuned transformers in terms of F1-score and recall, demonstrating generalizability beyond the training data. In comparison, SciBERT-based models exhibited higher precision, and BioBERT-based models exhibited higher recall, highlighting the importance of vocabulary and architecture. These models, representing the first enzyme NER algorithms, enable more effective enzyme text mining and information extraction. Codes for automated annotation and model generation are available from https://github.com/omicsNLP/enzymeNER and https://zenodo.org/doi/10.5281/zenodo.10581586.

Data augmentation and transfer learning for cross-lingual Named Entity Recognition in the biomedical domain

AbstractGiven the increase in production of data for the biomedical field and the unstoppable growth of the internet, the need for Information Extraction (IE) techniques has skyrocketed. Named Entity Recognition (NER) is one of such IE tasks useful for professionals in different areas. There are several settings where biomedical NER is needed, for instance, extraction and analysis of biomedical literature, relation extraction, organisation of biomedical documents, and knowledge-base completion. However, the computational treatment of entities in the biomedical domain has faced a number of challenges including its high cost of annotation, ambiguity, and lack of biomedical NER datasets in languages other than English. These difficulties have hampered data development, affecting both the domain itself and its multilingual coverage. The purpose of this study is to overcome the scarcity of biomedical data for NER in Spanish, for which only two datasets exist, by developing a robust bilingual NER model. Inspired by back-translation, this paper leverages the progress in Neural Machine Translation (NMT) to create a synthetic version of the Colorado Richly Annotated Full-Text (CRAFT) dataset in Spanish. Additionally, a new CRAFT dataset is constructed by replacing 20% of the entities in the original dataset generating a new augmented dataset. We evaluate two training methods: concatenation of datasets and continuous training to assess the transfer learning capabilities of transformers using the newly obtained datasets. The best performing NER system in the development set achieved an F-1 score of 86.39%. The novel methodology proposed in this paper presents the first bilingual NER system and it has the potential to improve applications across under-resourced languages.

Named entity recognition model based on Multi‐BiLSTM and competition mechanism

AbstractA named entity recognition (NER) model based on multiple bidirectional long short‐term memory networks (Multi‐BiLSTM) and competition mechanism (CM) is proposed. This model includes three parts: word vectorization module, learning module and application module. In the word vectorization module, the word vectors containing semantic features are generated based on BERT (Bidirectional Encoder Representations from Transformers). In the learning module, the multiple small‐scale BiLSTM are used to extract the contextual features of the word vectors synchronously. In the application module, the competition mechanism is adopted to select the relatively better models obtained in the learning module for recognizing entities. The results show the proposed model can reduce the parameters in the learning process, and the NER performance of the proposed model is better than that of the BERT‐BiLSTM‐CRF model. For example, the F1 can be increased by 1.02% and 0.39% for CoNLL2003 and Chinese resume datasets respectively.

Incorporating Entity Type-Aware and Word–Word Relation-Aware Attention in Generative Named Entity Recognition

Named entity recognition (NER) is a critical subtask in natural language processing. It is particularly valuable to gain a deeper understanding of entity boundaries and entity types when addressing the NER problem. Most previous sequential labeling models are task-specific, while recent years have witnessed the rise of generative models due to the advantage of tackling NER tasks in the encoder–decoder framework. Despite achieving promising performance, our pilot studies demonstrate that existing generative models are ineffective at detecting entity boundaries and estimating entity types. In this paper, a multiple attention framework is proposed which introduces the attention of entity-type embedding and word–word relation into the named entity recognition task. To improve the accuracy of entity-type mapping, we adopt an external knowledge base to calculate the prior entity-type distributions and then incorporate the information input to the model via the encoder’s self-attention. To enhance the contextual information, we take the entity types as part of the input. Our method obtains the other attention from the hidden states of entity types and utilizes it in self- and cross-attention mechanisms in the decoder. We transform the entity boundary information in the sequence into word–word relations and extract the corresponding embedding into the cross-attention mechanism. Through word–word relation information, the method can learn and understand more entity boundary information, thereby improving its entity recognition accuracy. We performed experiments on extensive NER benchmarks, including four flat and two long entity benchmarks. Our approach significantly improves or performs similarly to the best generative NER models. The experimental results demonstrate that our method can substantially enhance the capabilities of generative NER models.

Getting to Know Named Entity Recognition: Better Information Retrieval

Named entity recognition (NER) is a powerful computer system that utilizes various computing strategies to extract information from raw text input, since the early 1990s. With rapid advancement in AI and computing, NER models have gained significant attention and been serving as foundational tools across numerus professional domains to organize unstructured data for research and practical applications. This is particularly evident in the medical and healthcare fields, where NER models are essential in efficiently extract critical information from complex documents that are challenging for manual review. Despite its successes, NER present limitations in fully comprehending natural language nuances. However, the development of more advanced and user-friendly models promises to improve work experiences of professional users significantly.

Automated Spoken-to-Coded Pilot Weather Reports in General Aviation

A pilot weather report (PIREP) is a firsthand report of actual meteorological phenomena encountered by an aircraft in flight or on the ground. In general aviation (GA), PIREPs are a critical method of communicating weather information with pilots, controllers, and forecasters. However, GA pilots may find submitting PIREPs a cumbersome, manually intensive, and latent process. Automation of the radio PIREP submission process could lead to an increase in the quantity and quality (i.e., accuracy, timeliness, and usefulness) of GA PIREPs. This paper discusses the development and evaluation of a three-tier system to automatically convert spoken PIREPs to coded PIREPs using speech recognition, machine learning (ML), and natural-language-processing techniques. Specifically, this paper describes 1) the development, evaluation, and limitations of a trained commercial-of-the-shelf speech recognition system to generate text transcriptions of the spoken PIREPs, 2) an ML-based Named Entity Recognition model for segregating and labeling information from PIREP transcriptions, and 3) a final deterministic encoder to generate PIREP codes. If implemented in practice, this approach could significantly reduce interferences associated with submitting PIREPs, increase the number of PIREP submissions, improve the quality of PIREPs, and ultimately make flying safer for GA pilots.

Medical Named Entity Recognition Model Based on Knowledge Graph Enhancement

To improve the recognition ability of clinical named entity recognition (CNER) in a limited number of Chinese electronic medical records, it provides meaningful support for clinical advanced knowledge extraction. In this paper, using CCKS2019 Chinese electronic medical record as an experimental data source, a fusion model enhanced by knowledge graph (KG) is proposed, and the model is applied to specific Chinese CNER tasks. This study consists of three main parts: single-mode model construction and comparison experiment, KG enhancement experiment, and model fusion experiment. The model has achieved good performance in CNER from the results. The accuracy rate, recall rate, and F1 value are 83.825%, 84.705%, and 84.263%, respectively, which is the global optimal, which proves the effectiveness of the model. This provides a good help for further research of medical information.

ADAPTIVE DOMAIN-SPECIFIC NAMED ENTITY RECOGNITION METHOD WITH LIMITED DATA

The ever-evolving volume of digital information requires the development of innovative search strategies aimed at obtaining the necessary data efficiently and economically feasible. The urgency of the problem is emphasized by the growing complexity of information landscapes and the need for fast data extraction methodologies. In the field of natural language processing, named entity recognition (NER) is an essential task for extracting useful information from unstructured text input for further classification into predefined categories. Nevertheless, conventional methods frequently encounter difficulties when confronted with a limited amount of labeled data, posing challenges in real-world scenarios where obtaining substantial annotated datasets is problematic or costly. In order to address the problem of domain-specific NER with limited data, this work investigates NER techniques that can overcome these constraints by continuously learning from newly collected information on pre-trained models. Several techniques are also used for making the greatest use of the limited labeled data, such as using active learning, exploiting unlabeled data, and integrating domain knowledge. Using domain-specific datasets with different levels of annotation scarcity, the fine-tuning process of pre-trained models, such as transformer-based models (TRF) and Toc2Vec (token-to-vector) models is investigated. The results show that, in general, expanding the volume of training data enhances most models' performance for NER, particularly for models with sufficient learning ability. Depending on the model architecture and the complexity of the entity label being learned, the effect of more data on the model's performance can change. After increasing the training data by 20%, the LT2V model shows the most balanced growth in accuracy overall by 11% recognizing 73% of entities and processing speed. Meanwhile, with consistent processing speed and the greatest F1-score, the Transformer-based model (TRF) shows promise for effective learning with less data, achieving 74% successful prediction and a 7% increase in performance after expanding the training data to 81%. Our results pave the way for the creation of more resilient and efficient NER systems suited to specialized domains and further the field of domain-specific NER with sparse data. We also shed light on the relative merits of various NER models and training strategies, and offer perspectives for future research.

A false emotion opinion target extraction model with two stage BERT and background information fusion

As social media has gradually become an indispensable part of people’s life, more and more users begin to express their opinions on social media. These opinions contain rich emotional information, as well as many abnormal false emotion. And the opinion target rich in false emotion will have great potential value in the field of movie box office prediction, public opinion guidance and so on. Therefore, we propose a false emotion opinion target extraction task and two stage BERT+CRF model based on the concept of false emotion. Firstly, two stage BERT network includes C-BERT and W-BERT. We use C-BERT to learn the users comment feature, and W-BERT to learn the background content features of microblog. Then, we design five feature interaction fusion methods to fuse the features learned by C-BERT and W-BERT. At last, we transform the false emotion opinion target extraction task into a sequence labeling task, and use the CRF method to learn the label dependency of the context and output the optimal sequence among all candidate sequences. Due to the lack of false emotion opinion target datasets, we collate a false emotion opinion target dataset Weibo23. Comparing with the advanced models used in related research, including some sequence labeling models and named entity recognition models, two stage BERT+CRF model achieves an F1 score of 0.8834, which is about 10%–20% higher.

A Method for Ancient Book Named Entity Recognition Based on BERT-Global Pointer

Correct identification of entities in ancient books and documents is the basic step of analyzing ancient Chinese texts, and provides an important prerequisite for in-depth mining of humanistic knowledge in ancient books and documents. In CCL2023 named entity recognition task of ancient books, according to the task definition and the re-quirements of the task organizer, this paper proposes the BERT Global Pointer named entity recognition model; Fine tune the field adaptation training based on the unlabeled 24 history ancient book text data; SWA, FGM, cross validation and post-processing are used to improve the recognition accuracy of the model. The experimental results show that the model and the strategy proposed in this paper have good recognition effect in the multi dynasties, cross domain ancient book entity recognition scene. F1 value on the final line reaches 95.083%.

SENCR: A Span Enhanced Two-Stage Network with Counterfactual Rethinking for Chinese NER

Recently, lots of works that incorporate external lexicon information into character-level Chinese named entity recognition(NER) to overcome the lackness of natural delimiters of words, have achieved many advanced performance. However, obtaining and maintaining high-quality lexicons is costly, especially in special domains. In addition, the entity boundary bias caused by high mention coverage in some boundary characters poses a significant challenge to the generalization of NER models but receives little attention in the existing literature. To address these issues, we propose SENCR, a Span Enhanced Two-stage Network with Counterfactual Rethinking for Chinese NER, that contains a boundary detector for boundary supervision, a convolution-based type classifier for better span representation and a counterfactual rethinking(CR) strategy for debiased boundary detection in inference. The proposed boundary detector and type classifier are jointly trained with the same contextual encoder and then the trained boundary detector is debiased by our proposed CR strategy without modifying any model parameters in the inference stage. Extensive experiments on four Chinese NER datasets show the effectiveness of our proposed approach.

Robust Few-Shot Named Entity Recognition with Boundary Discrimination and Correlation Purification

Few-shot named entity recognition (NER) aims to recognize novel named entities in low-resource domains utilizing existing knowledge. However, the present few-shot NER models assume that the labeled data are all clean without noise or outliers, and there are few works focusing on the robustness of the cross-domain transfer learning ability to textual adversarial attacks in Few-shot NER. In this work, we comprehensively explore and assess the robustness of few-shot NER models under textual adversarial attack scenario, and found the vulnerability of existing few-shot NER models. Furthermore, we propose a robust two-stage few-shot NER method with Boundary Discrimination and Correlation Purification (BDCP). Specifically, in the span detection stage, the entity boundary discriminative module is introduced to provide a highly distinguishing boundary representation space to detect entity spans. In the entity typing stage, the correlations between entities and contexts are purified by minimizing the interference information and facilitating correlation generalization to alleviate the perturbations caused by textual adversarial attacks. In addition, we construct adversarial examples for few-shot NER based on public datasets Few-NERD and Cross-Dataset. Comprehensive evaluations on those two groups of few-shot NER datasets containing adversarial examples demonstrate the robustness and superiority of the proposed method.

ConsistNER: Towards Instructive NER Demonstrations for LLMs with the Consistency of Ontology and Context

Named entity recognition (NER) aims to identify and classify specific entities mentioned in textual sentences. Most existing superior NER models employ the standard fully supervised paradigm, which requires a large amount of annotated data during training. In order to maintain performance with insufficient annotation resources (i.e., low resources), in-context learning (ICL) has drawn a lot of attention, due to its plug-and-play nature compared to other methods (e.g., meta-learning and prompt learning). In this manner, how to retrieve high-correlated demonstrations for target sentences serves as the key to emerging ICL ability. For the NER task, the correlation implies the consistency of both ontology (i.e., generalized entity type) and context (i.e., sentence semantic), which is ignored by previous NER demonstration retrieval techniques. To address this issue, we propose ConsistNER, a novel three-stage framework that incorporates ontological and contextual information for low-resource NER. Firstly, ConsistNER employs large language models (LLMs) to pre-recognize potential entities in a zero-shot manner. Secondly, ConsistNER retrieves the sentence-specific demonstrations for each target sentence based on the two following considerations: (1) Regarding ontological consistency, demonstrations are filtered into a candidate set based on ontology distribution. (2) Regarding contextual consistency, an entity-aware self-attention mechanism is introduced to focus more on the potential entities and semantic-correlated tokens. Finally, ConsistNER feeds the retrieved demonstrations for all target sentences into LLMs for prediction. We conduct experiments on four widely-adopted NER datasets, including both general and specific domains. Experimental results show that ConsistNER achieves a 6.01%-26.37% and 3.07%-21.18% improvement over the state-of-the-art baselines on Micro-F1 scores under 1- and 5-shot settings, respectively.

Three Heads Are Better than One: Improving Cross-Domain NER with Progressive Decomposed Network

Cross-domain named entity recognition (NER) tasks encourage NER models to transfer knowledge from data-rich source domains to sparsely labeled target domains. Previous works adopt the paradigms of pre-training on the source domain followed by fine-tuning on the target domain. However, these works ignore that general labeled NER source domain data can be easily retrieved in the real world, and soliciting more source domains could bring more benefits. Unfortunately, previous paradigms cannot efficiently transfer knowledge from multiple source domains. In this work, to transfer multiple source domains' knowledge, we decouple the NER task into the pipeline tasks of mention detection and entity typing, where the mention detection unifies the training object across domains, thus providing the entity typing with higher-quality entity mentions. Additionally, we request multiple general source domain models to suggest the potential named entities for sentences in the target domain explicitly, and transfer their knowledge to the target domain models through the knowledge progressive networks implicitly. Furthermore, we propose two methods to analyze in which source domain knowledge transfer occurs, thus helping us judge which source domain brings the greatest benefit. In our experiment, we develop a Chinese cross-domain NER dataset. Our model improved the F1 score by an average of 12.50% across 8 Chinese and English datasets compared to models without source domain data.

The Integration of Complex Systems Science and Community-Based Research: A Scoping Review

Complex systems science (CSS) and community-based research (CBR) have emerged over the past 50 years as complementary disciplines. However, there is a gap in understanding what has driven the recent proliferation of integrating these two disciplines to study complex and relevant issues. In this review, we report on the results of a scoping review of articles that utilized both disciplines. After two levels of reviewing articles using DistillerSR, a web-based platform designed to streamline and facilitate the process of conducting systematic reviews, we used two forms of natural language processing to extract data. We developed a novel named entity recognition model to extract descriptive information from the corpus of articles. We also conducted dynamic topic modeling to deductively examine in tandem the development of CSS and CBR and to inductively discover the specific topics that may be driving their use in research and practice. We find that among the CSS and CBR papers, CBR topic frequency has grown at a faster pace than CSS, with CBR using CSS concepts and techniques more often. Four topics that may be driving this trend are collaboration within social systems, business management, food and land use and knowledge, and water shed management. We conclude by discussing the implications of this work for researchers and practitioners who are interested in studying and solving complex social, economic, and health-related issues.

Contribution Analysis of Large Language Models and Data Augmentations for Person Names in Solving Legal Bar Examination at COLIEE 2023

This paper describes our system for COLIEE 2023 Task 4, which automatically answers Japanese legal bar exam problems. We propose an extension to our previous system in COLIEE 2022, which achieved the highest accuracy among all submissions using data augmentation. We focus on problems that include mentions of person names. In this paper, we present two main contributions. First, we incorporate LUKE as our deep learning component, which is a named entity recognition model trained on RoBERTa. Second, we fine-tune the pretrained LUKE model in multiple ways, comparing fine-tuning on training datasets that include alphabetical person names and ensembling different fine-tuning models. We confirmed that LUKE and its fine-tuned model on person type problems improve their accuracies. Our formal run results show that LUKE and our fine-tuning approach using alphabetical person names were effective, achieving an accuracy of 0.69 in the COLIEE 2023 Task 4 formal run.

A Chinese named entity recognition model incorporating recurrent cell and information state recursion

Chinese is characterized by high syntactic complexity, chaotic annotation granularity, and slow convergence. Joint learning models can effectively improve the accuracy of Chinese Named Entity Recognition (NER), but they focus too much on local feature information and reduce the ability of long sequence feature extraction. To address the limitations of long sequence feature extraction ability, we propose a Chinese NER model called Incorporating Recurrent Cell and Information State Recursion (IRCSR-NER). The model integrates recurrent cells and information state recursion to improve the recognition ability of long entity boundaries. To solve the problem that Chinese and English have different focuses in syntactic analysis. We use the syntactic dependency approach to add lexical relationship information to sentences represented at the word level. The IRCSR-NER is applied to sequence feature extraction to improve the model efficiency and long-text feature extraction ability. The model captures contextual long-distance dependent information while focusing on local feature information. We evaluated our proposed model using four public datasets and compared it with other mainstream models. Experimental results demonstrate that our model outperforms traditional and mainstream models.

Implementing deep learning-based named entity recognition for obtaining narcotics abuse data in Indonesia

<span lang="EN-US">The availability of drug abuse data from the official website of the National Narcotics Board of Indonesia is not up-to-date. Besides, the drug reports from Indonesian National Narcotics Board are only published once a year. This study aims to utilize online news sites as a data source for collecting information about drug abuse in Indonesia. In addition, this study also builds a named entity recognition (NER) model to extract information from news texts. The primary NER model in this study uses the convolutional neural network-long short-term memory (CNNs-LSTM) architecture because it can produce a good performance and only requires a relatively short computation time. Meanwhile, the baseline NER model uses the bidirectional long short-term memory-conditional random field (Bi-LSTMs-CRF) architecture because it is easy to implement using the Flair framework. The primary model that has been built results in a performance (F1 score) of 82.54%. Meanwhile, the baseline model only results in a performance (F1 score) of 69.67%. Then, the raw data extracted by NER is processed to produce the number of drug suspects in Indonesia from 2018-2020. However, the data that has been produced is not as complete as similar data sourced from Indonesian National Narcotics Board publications.</span>

Evaluating Natural Language Processing Packages for Predicting Hospital-Acquired Pressure Injuries From Clinical Notes.

Incidence of hospital-acquired pressure injury, a key indicator of nursing quality, is directly proportional to adverse outcomes, increased hospital stays, and economic burdens on patients, caregivers, and society. Thus, predicting hospital-acquired pressure injury is important. Prediction models use structured data more often than unstructured notes, although the latter often contain useful patient information. We hypothesize that unstructured notes, such as nursing notes, can predict hospital-acquired pressure injury. We evaluate the impact of using various natural language processing packages to identify salient patient information from unstructured text. We use named entity recognition to identify keywords, which comprise the feature space of our classifier for hospital-acquired pressure injury prediction. We compare scispaCy and Stanza, two different named entity recognition models, using unstructured notes in Medical Information Mart for Intensive Care III, a publicly available ICU data set. To assess the impact of vocabulary size reduction, we compare the use of all clinical notes with only nursing notes. Our results suggest that named entity recognition extraction using nursing notes can yield accurate models. Moreover, the extracted keywords play a significant role in the prediction of hospital-acquired pressure injury.

Application of BiLSTM-CRF model with different embeddings for product name extraction in unstructured Turkish text

Named entity recognition (NER) plays a pivotal role in Natural Language Processing by identifying and classifying entities within textual data. While NER methodologies have seen significant advancements, driven by pretrained word embeddings and deep neural networks, the majority of these studies have focused on text with well-defined grammar and structure. A significant research gap exists concerning NER in informal or unstructured text, where traditional grammar rules and sentence structure are absent. This research addresses this crucial gap by focusing on the detection of product names within unstructured Turkish text. To accomplish this, we propose a deep learning-based NER model which combines a Bidirectional Long Short-Term Memory (BiLSTM) architecture with a Conditional Random Field (CRF) layer, further enhanced by FastText embeddings. To comprehensively evaluate and compare our model’s performance, we explore different embedding approaches, including Word2Vec and Glove, in conjunction with the Bidirectional Long Short-Term Memory and Conditional Random Field (BiLSTM-CRF) model. Furthermore, we conduct comparisons against BERT to assess the efficacy of our approach. Our experimentation utilizes a Turkish e-commerce dataset gathered from the internet, where traditional grammatical and structural rules may not apply. The BiLSTM-CRF model with FastText embeddings achieved an F1 score value of 57.40%, a precision value of 55.78%, and a recall value of 59.12%. These results indicate promising performance in outperforming other baseline techniques. This research contributes to the field of NER by addressing the unique challenges posed by unstructured Turkish text and opens avenues for improved entity recognition in informal language settings, with potential applications across various domains.