Named Entity Recognition Methods Research Articles

Biomedical named entity recognition using improved green anaconda-assisted Bi-GRU-based hierarchical ResNet model

BackgroundBiomedical text mining is a technique that extracts essential information from scientific articles using named entity recognition (NER). Traditional NER methods rely on dictionaries, rules, or curated corpora, which may not always be accessible. To overcome these challenges, deep learning (DL) methods have emerged. However, DL-based NER methods may need help identifying long-distance relationships within text and require significant annotated datasets.ResultsThis research has proposed a novel model to address the challenges in natural language processing. The Improved Green anaconda-assisted Bi-GRU based Hierarchical ResNet BNER model (IGa-BiHR BNERM) is the model. IGa-BiHR BNERM model has shown promising results in accurately identifying named entities. The MACCROBAT dataset was obtained from Kaggle and underwent several pre-processing steps such as Stop Word Filtering, WordNet processing, Removal of non-alphanumeric characters, stemming Segmentation, and Tokenization, which is standardized and improves its quality. The pre-processed text was fed into a feature extraction model like the Robustly Optimized BERT –Whole Word Masking model. This model provides word embeddings with semantic information. Then, the BNER process utilized an Improved Green Anaconda-assisted Bi-GRU-based Hierarchical ResNet BNER model (IGa-BiHR BNERM).ConclusionTo improve the training phase of the IGa-BiHR BNERM, the Improved Green Anaconda Optimization technique was used to select optimal weight parameter coefficients for training the model parameters. After the model was tested using the MACCROBAT dataset, it outperformed previous models with a tremendous accuracy rate of 99.11%. This model effectively and accurately identifies biomedical names within the text, significantly advancing this field.

Template-Free Prompting for Few-Shot Named Entity Recognition via Semantic-Enhanced Contrastive Learning.

Prompt tuning has achieved great success in various sentence-level classification tasks by using elaborated label word mappings and prompt templates. However, for solving token-level classification tasks, e.g., named entity recognition (NER), previous research, which utilizes N-gram traversal for prompting all spans with all possible entity types, is time-consuming. To this end, we propose a novel prompt-based contrastive learning method for few-shot NER without template construction and label word mappings. First, we leverage external knowledge to initialize semantic anchors for each entity type. These anchors are simply appended with input sentence embeddings as template-free prompts (TFPs). Then, the prompts and sentence embeddings are in-context optimized with our proposed semantic-enhanced contrastive loss. Our proposed loss function enables contrastive learning in few-shot scenarios without requiring a significant number of negative samples. Moreover, it effectively addresses the issue of conventional contrastive learning, where negative instances with similar semantics are erroneously pushed apart in natural language processing (NLP)-related tasks. We examine our method in label extension (LE), domain-adaption (DA), and low-resource generalization evaluation tasks with six public datasets and different settings, achieving state-of-the-art (SOTA) results in most cases.

Vocabulary-Enhanced Named Entity Recognition and its Application on Distribution Network Maintenance

Named entity recognition is a crucial task in natural language processing, serving downstream tasks such as information extraction and question-answering systems. With the success of Transformer in the field of natural language processing, named entity recognition has made significant strides. However, despite these advancements, the recognition capability in specialized domains remains challenging. The distribution network system contains a vast amount of equipment maintenance text, characterized by inherent complexity and specialization, lacking effective recognition and extraction methods. To address this issue, we propose a vocabulary-enhanced named entity recognition method. Unlike the original named entity recognition methods, we employ a vocabulary adapter to enhance the understanding of text semantic information by integrating matching information between the dictionary and the text. This method strengthens the ability to extract specialized nouns. In conjunction with the TaCL-BERT pre-trained model, we conducted training on our self-constructed distribution network maintenance dataset, GNP-M, to better align the model with the context and specific entities of the electrical grid maintenance domain. Simultaneously, we compared our method with four other named entity recognition models, revealing satisfactory performance results.

PromptCNER: A Segmentation-based Method for Few-shot Chinese NER with Prompt-tuning

Recognizing Chinese entities in low-resource settings is a challenging but promising task, which extracts structured pre-defined entities and corresponding types from unstructured text. Compared with the prosperous Named Entity Recognition (NER) methods for Indo-European languages, such as English, the research on Chinese NER is still in its infancy. The main obstacles to the development of Chinese NER methods include the ambiguity of Chinese entity boundary recognition and limited data resources. To address these issues, in this paper, a word-segmentation-based model is present for few-shot Chinese NER. First, we enumerate all possible candidate entity spans on the character level for accurate entity boundary identification with the proposed word segmentation and combination strategy. Then, one kind of question-answer-based prompt template loaded with the candidate entity spans is proposed to cast entity extraction into the masked token prediction task, for dealing with the low-data problem by taking full advantage of the generality and transferability of the pre-trained language model. The extensive experimental results show that our method outperforms the state-of-the-art baselines in low-data settings and also achieves comparable performance in full-data settings.

A Chinese named entity recognition method for landslide geological disasters based on deep learning

Landslide Named Entity Recognition (LNER) involves extracting specific entities from Chinese unstructured landslide disaster texts, which is crucial for constructing a knowledge graph and supporting landslide prevention efforts. This study proposes a deep learning-based LNER model that utilizes Bidirectional Encoder Representations from Transformer (BERT) for word embeddings and integrates the Conditional Random Fields (CRF) algorithm and projected gradient descent (PGD) adversarial neural networks to enhance sequence labeling accuracy. The practical implications of this research lie in improving the efficiency and precision of disaster information extraction, aiding in real-time decision-making and risk mitigation strategies. Experiments on the constructed dataset show that the model effectively identifies eight types of landslide entities, achieving a highest F1 score of 89.7%.

A Unified Model for Chinese Cyber Threat Intelligence Flat Entity and Nested Entity Recognition

In recent years, as cybersecurity threats have become increasingly severe and cyberattacks have occurred frequently, higher requirements have been put forward for cybersecurity protection. Therefore, the Named Entity Recognition (NER) technique, which is the cornerstone of Cyber Threat Intelligence (CTI) analysis, is particularly important. However, most existing NER studies are limited to recognizing single-layer flat entities, ignoring the possible nested entities in CTI. On the other hand, most of the existing studies focus on English CTIs, and the existing models performed poorly in a limited number of Chinese CTI studies. Given the above challenges, we propose in this paper a novel unified model, RBTG, which aims to identify flat and nested entities in Chinese CTI effectively. To overcome the difficult boundary recognition problem and the direction-dependent and distance-dependent properties in Chinese CTI NER, we use Global Pointer as the decoder and TENER as the encoder layer, respectively. Specifically, the Global Pointer layer solves the problem of the insensitivity of general NER methods to entity boundaries by utilizing the relative position information and the multiplicative attention mechanism. The TENER layer adapts to the Chinese CTI NER task by introducing an attention mechanism with direction awareness and distance awareness. Meanwhile, to cope with the complex feature capture of hierarchical structure and dependencies among Chinese CTI nested entities, the TENER layer solves the problem by following the structure of multiple self-attention layers and feed-forward network layers superimposed on each other in the Transformer. In addition, to fill the gap in the Chinese CTI nested entity dataset, we further apply the Large Language Modeling (LLM) technique and domain knowledge to construct a high-quality Chinese CTI nested entity dataset, CDTinee, which consists of six entity types selected from STIX, including nearly 4000 entity types extracted from more than 3000 threatening sentences. In the experimental session, we conduct extensive experiments on multiple datasets, and the results show that the proposed model RBTG outperforms the baseline model in both flat NER and nested NER.

A multimodal approach for few-shot biomedical named entity recognition in low-resource languages

In this study, we revisit named entity recognition (NER) in the biomedical domain from a multimodal perspective, with a particular focus on applications in low-resource languages. Existing research primarily relies on unimodal methods for NER, which limits the potential for capturing diverse information. To address this limitation, we propose a novel method that integrates a cross-modal generation module to transform unimodal data into multimodal data, thereby enabling the use of enriched multimodal information for NER. Additionally, we design a cross-modal filtering module to mitigate the adverse effects of text-image mismatches in multimodal NER. We validate our proposed method on two biomedical datasets specifically curated for low-resource languages. Experimental results demonstrate that our method significantly enhances the performance of NER, highlighting its effectiveness and potential for broader applications in biomedical research and low-resource language contexts.

Dual Contrastive Learning for Cross-Domain Named Entity Recognition

Benefiting many information retrieval applications, named entity recognition (NER) has shown impressive progress. Recently, there has been a growing trend to decompose complex NER tasks into two subtasks (e.g., entity span detection (ESD) and entity type classification (ETC), to achieve better performance. Despite the remarkable success, from the perspective of representation, existing methods do not explicitly distinguish non-entities and entities, which may lead to ESD errors. Meanwhile, they do not explicitly distinguish entities with different entity types, which may lead to entity type misclassification. As such, the limited representation abilities may challenge some competitive NER methods, leading to unsatisfactory performance, especially in the low-resource setting (e.g., cross-domain NER). In light of these challenges, we propose to utilize contrastive learning to refine the original chaotic representations and learn the generalized representations for cross-domain NER. In particular, this article proposes a dual contrastive learning model (Dual-CL), which respectively utilizes a token-level contrastive learning module and a sentence-level contrastive learning module to enhance ESD, ETC for cross-domain NER. Empirical results on 10 domain pairs under two different settings show that Dual-CL achieves better performances than compared baselines in terms of several standard metrics. Moreover, we conduct detailed analyses to are presented to better understand each component’s effectiveness.

HiNER: Hierarchical feature fusion for Chinese named entity recognition

Named Entity Recognition (NER) aims to extract structured entity information from unstructured textual data by identifying entity boundaries and categories. Chinese NER is more challenging than that of English due to the complex structure and ambiguous word boundaries, as well as nested and discontinuous occurrences of entities. Previous Chinese NER methods are limited by their character-based approach and dependence on external lexical information, which is often non-contextualized, leading to the introduction of noise and potentially compromising model performance. This paper proposes a novel Chinese NER model, HiNER, which leverages external semantic enhancement and hierarchical attention fusion. Specifically, we initially formulate the Chinese NER as a character–character relation classification task, thoroughly taking into account the cases of nested and discontinuous entities. Then, by incorporating syntactic information, we develop a Triformer module that is used to better integrate Chinese character, lexical, and syntactic embeddings, carefully considering the impact of external semantic enhancement on the original text embeddings and reducing extrinsic information interference to some extent. In addition, through the fusion of local and global attention mechanisms, the representation of character–character relationships is enhanced, allowing for the effective capture of semantic features at various hierarchical levels within the Chinese context. We conduct extensive experiments on seven Chinese NER datasets, and the results indicate that the HiNER model achieves state-of-the-art (SOTA) performance. The outcomes also confirm that external semantic enhancement and hierarchical attention fusion can provide better assistance in accomplishing the Chinese NER task.

WITHDRAWN: TourismNER: A tourism named entity recognition method based on entity boundary joint prediction

Tourism named entity recognition is indispensable in tourism information extraction, and plays a crucial role in constructing tourism knowledge map and enhancing tourism knowledge quiz system. The difficulty of tourism named entity recognition lies in its complex nested structure, and the lengthy entity naming length. To address these existing problems, we propose a tourism named entity recognition model that jointly predicts entity boundaries, adopting a training strategy of data preprocessing to enhance the model’s ability for tourism named entity boundary recognition, while our model introduces a pre-trained Bert model as well as BiLSTM coding to enhance the representation of the model’s contexts, and uses a combined predictor of Biaffine and MLP to enhance the model’s recognition performance for boundaries, as well as introducing label smoothing cross entropy to smooth the target labels during the training process. Experiments are conducted on three datasets with different granularities. From the analysis of the experimental results, it can be seen that the named entity recognition method achieves higher accuracy and F1 value compared with the optimal baseline model, and also proves the effectiveness and generality of the modeling method proposed in this paper.

Chinese EMR Named Entity Recognition Using Fused Label Relations Based on Machine Reading Comprehension Framework.

Chinese electronic medical record (EMR) presents significant challenges for named entity recognition (NER) due to their specialized nature, unique language features, and diverse expressions. Traditionally, NER is treated as a sequence labeling task, where each token is assigned a label. Recent research has reframed NER within the machine reading comprehension (MRC) framework, extracting entities in a question-answer format, achieving state-of-the-art performance. However, these MRC-based methods have a significant limitation: they extract entities of various types independently, ignoring their interrelations. To address this, we introduce the Fusion Label Relations with MRC (FLR-MRC) model, which enhances the MRC model by implicitly capturing dependencies among entity types. FLR-MRC models interrelations between labels using graph attention networks, integrating these with textual data to identify entities. On the benchmark CMeEE and CCKS2017-CNER datasets, FLR-MRC achieves F1-scores of 0.6652 and 0.9101, respectively, outperforming existing clinical NER methods.

Improving dictionary-based named entity recognition with deep learning.

Dictionary-based named entity recognition (NER) allows terms to be detected in a corpus and normalized to biomedical databases and ontologies. However, adaptation to different entity types requires new high-quality dictionaries and associated lists of blocked names for each type. The latter are so far created by identifying cases that cause many false positives through manual inspection of individual names, a process that scales poorly. In this work, we aim to improve block list s by automatically identifying names to block, based on the context in which they appear. By comparing results of three well-established biomedical NER methods, we generated a dataset of over 12.5 million text spans where the methods agree on the boundaries and type of entity tagged. These were used to generate positive and negative examples of contexts for four entity types (genes, diseases, species, and chemicals), which were used to train a Transformer-based model (BioBERT) to perform entity type classification. Application of the best model (F1-score = 96.7%) allowed us to generate a list of problematic names that should be blocked. Introducing this into our system doubled the size of the previous list of corpus-wide blocked names. In addition, we generated a document-specific list that allows ambiguous names to be blocked in specific documents. These changes boosted text mining precision by ∼5.5% on average, and over 8.5% for chemical and 7.5% for gene names, positively affecting several biological databases utilizing this NER system, like the STRING database, with only a minor drop in recall (0.6%). All resources are available through Zenodo https://doi.org/10.5281/zenodo.11243139 and GitHub https://doi.org/10.5281/zenodo.10289360.

Transformer-based Named Entity Recognition for Clinical Cancer Drug Toxicity by Positive-unlabeled Learning and KL Regularizers

Background: With increasing rates of polypharmacy, the vigilant surveillance of clinical drug toxicity has emerged as an important With increasing rates of polypharmacy, the vigilant surveillance of clinical drug toxicity has emerged as an important concern. Named Entity Recognition (NER) stands as an indispensable undertaking, essential for the extraction of valuable insights regarding drug safety from the biomedical literature. In recent years, significant advancements have been achieved in the deep learning models on NER tasks. Nonetheless, the effectiveness of these NER techniques relies on the availability of substantial volumes of annotated data, which is labor-intensive and inefficient. Methods: This study introduces a novel approach that diverges from the conventional reliance on manually annotated data. It employs a transformer-based technique known as Positive-Unlabeled Learning (PULearning), which incorporates adaptive learning and is applied to the clinical cancer drug toxicity corpus. To improve the precision of prediction, we employ relative position embeddings within the transformer encoder. Additionally, we formulate a composite loss function that integrates two Kullback-Leibler (KL) regularizers to align with PULearning assumptions. The outcomes demonstrate that our approach attains the targeted performance for NER tasks, solely relying on unlabeled data and named entity dictionaries. Conclusion: Our model achieves an overall NER performance with an F1 of 0.819. Specifically, it attains F1 of 0.841, 0.801 and 0.815 for DRUG, CANCER, and TOXI entities, respectively. A comprehensive analysis of the results validates the effectiveness of our approach in comparison to existing PULearning methods on biomedical NER tasks. Additionally, a visualization of the associations among three identified entities is provided, offering a valuable reference for querying their interrelationships.

Class-Imbalanced-Aware Distantly Supervised Named Entity Recognition.

Distantly supervised named entity recognition (NER), which automatically learns NER models without manually labeling data, has gained much attention recently. In distantly supervised NER, positive unlabeled (PU) learning methods have achieved notable success. However, existing PU learning-based NER methods are unable to automatically handle the class imbalance and further depend on the estimation of the unknown class prior; thus, the class imbalance and imperfect estimation of the class prior degenerate the NER performance. To address these issues, this article proposes a novel PU learning method for distantly supervised NER. The proposed method can automatically handle the class imbalance and does not need to engage in class prior estimation, which enables the proposed methods to achieve the state-of-the-art performance. Extensive experiments support our theoretical analysis and validate the superiority of our method.

A novel prompting method for few-shot NER via LLMs

In various natural language processing tasks, significant strides have been made by Large Language Models (LLMs). Researchers leverage prompt method to conduct LLMs in accomplishing specific tasks under few-shot conditions. However, the prevalent use of LLMs’ prompt methods mainly focuses on guiding generative tasks, and employing existing prompts may result in poor performance in Named Entity Recognition (NER) tasks. To tackle this challenge, we propose a novel prompting method for few-shot NER. By enhancing existing prompt methods, we devise a standardized prompts tailored for the utilization of LLMs in NER tasks. Specifically, we structure the prompts into three components: task definition, few-shot demonstration, and output format. The task definition conducts LLMs in performing NER tasks, few-shot demonstration assists LLMs in understanding NER task objectives through specific output demonstration, and output format restricts LLMs’ output to prevent the generation of unnecessary results. The content of these components has been specifically tailored for NER tasks. Moreover, for the few-shot demonstration within the prompts, we propose a selection strategy that utilizes feedback from LLMs’ outputs to identify more suitable few-shot demonstration as prompts. Additionally, to enhance entity recognition performance, we enrich the prompts by summarizing error examples from the output process of LLMs and integrating them as additional prompts.

A New Chinese Named Entity Recognition Method for Pig Disease Domain Based on Lexicon-Enhanced BERT and Contrastive Learning

A New Chinese Named Entity Recognition Method for Pig Disease Domain Based on Lexicon-Enhanced BERT and Contrastive Learning

Research on named entity recognition method for high-speed railway technology transformation project text

Abstract High-speed railway technology transformation projects are crucial to the layout of high-speed rail transportation production, high-speed rail safety, and high-speed rail operation and management. However, the data of high-speed railway technology transformation projects is mainly stored in the form of unstructured text, and the relevant personnel of high-speed railroad often need to manually go through the form to extract the key information of the project data and enter it into the system, with a low degree of automation and intelligence. Therefore, to improve the efficiency and quality of processing the data of high-speed railway technology transformation projects, and help the intelligent development of high-speed railways, this paper constructs the RoBERTa-BiLSTM-CRF fusion model to automate the identification of key information entities in the data of high-speed railway technology transformation project. The experimental data were selected as technology transformation project documents of a railway bureau from 2013 to 2022, and six defined entity types were identified. Experimental results show that the constructed model can effectively identify six types of defined entities, and the F1 value of each type of entity is above 90%. In addition, compared with other commonly used models, the F1 value has been significantly improved. The fusion model proposed in this paper can be used as a method to automatically extract key information for high-speed rail technology transformation projects.

Application of machine reading comprehension techniques for named entity recognition in materials science

Materials science is an interdisciplinary field that studies the properties, structures, and behaviors of different materials. A large amount of scientific literature contains rich knowledge in the field of materials science, but manually analyzing these papers to find material-related data is a daunting task. In information processing, named entity recognition (NER) plays a crucial role as it can automatically extract entities in the field of materials science, which have significant value in tasks such as building knowledge graphs. The typically used sequence labeling methods for traditional named entity recognition in material science (MatNER) tasks often fail to fully utilize the semantic information in the dataset and cannot effectively extract nested entities. Herein, we proposed to convert the sequence labeling task into a machine reading comprehension (MRC) task. MRC method effectively can solve the challenge of extracting multiple overlapping entities by transforming it into the form of answering multiple independent questions. Moreover, the MRC framework allows for a more comprehensive understanding of the contextual information and semantic relationships within materials science literature, by integrating prior knowledge from queries. State-of-the-art (SOTA) performance was achieved on the Matscholar, BC4CHEMD, NLMChem, SOFC, and SOFC-Slot datasets, with F1-scores of 89.64%, 94.30%, 85.89%, 85.95%, and 71.73%, respectively in MRC approach. By effectively utilizing semantic information and extracting nested entities, this approach holds great significance for knowledge extraction and data analysis in the field of materials science, and thus accelerating the development of material science.Scientific contributionWe have developed an innovative NER method that enhances the efficiency and accuracy of automatic entity extraction in the field of materials science by transforming the sequence labeling task into a MRC task, this approach provides robust support for constructing knowledge graphs and other data analysis tasks.

Improved XLNet modeling for Chinese named entity recognition of edible fungus.

The diversity of edible fungus species and the extent of mycological knowledge pose significant challenges to the research, cultivation, and popularization of edible fungus. To tackle this challenge, there is an urgent need for a rapid and accurate method of acquiring relevant information. The emergence of question and answer (Q&A) systems has the potential to solve this problem. Named entity recognition (NER) provides the basis for building an intelligent Q&A system for edible fungus. In the field of edible fungus, there is a lack of a publicly available Chinese corpus suitable for use in NER, and conventional methods struggle to capture long-distance dependencies in the NER process. This paper describes the establishment of a Chinese corpus in the field of edible fungus and introduces an NER method for edible fungus information based on XLNet and conditional random fields (CRFs). Our approach combines an iterated dilated convolutional neural network (IDCNN) with a CRF. First, leveraging the XLNet model as the foundation, an IDCNN layer is introduced. This layer addresses the limited capacity to capture features across utterances by extending the receptive field of the convolutional kernel. The output of the IDCNN layer is input to the CRF layer, which mitigates any labeling logic errors, resulting in the globally optimal labels for the NER task relating to edible fungus. Experimental results show that the precision achieved by the proposed model reaches 0.971, with a recall of 0.986 and an F1-score of 0.979. The proposed model outperforms existing approaches in terms of these evaluation metrics, effectively recognizing entities related to edible fungus information and offering methodological support for the construction of knowledge graphs.

Chinese Named Entity Recognition method based on multi-feature fusion and biaffine

Chinese Named Entity Recognition (CNER) focuses on precisely identifying predefined structural categories in unstructured Chinese text. Most existing CNER models do not consider the unique glyph and pinyin features of Chinese characters, but the rich semantic features hidden behind these features have a good effect on enhancing the judgment ability of language models. At the same time, it is difficult to identify the boundaries of Chinese nested entities, and accurately identifying the boundaries of entities within nested entities is also a difficult problem to solve. We propose a CNER method based on multi-feature fusion technology and biaffine mechanism to address the above issues: In the input representation layer, integrate the glyph and pinyin features of Chinese characters together, intuitively capturing the semantics of Chinese characters. Furthermore, biaffine mechanism has been introduced to provide a comprehensive view of the input on a global scale. This mechanism effectively converts the task of entity recognition into a problem of assigning scores to spans, hence enhancing the precision of identifying entity borders. In order to evaluate the efficacy of the model, a series of experiments were done on three Chinese entity recognition datasets: Resume, MSRA, and People Daily. The experimental results show that the solid boundary can be identified more accurately, and the F1 values of 96.49%, 96.26% and 96.19% are obtained respectively, which has a better recognition effect than the baseline model.