Named Entity Recognition Methods Research Articles

GPT for medical entity recognition in Spanish

AbstractIn recent years, there has been a remarkable surge in the development of Natural Language Processing (NLP) models, particularly in the realm of Named Entity Recognition (NER). Models such as BERT have demonstrated exceptional performance, leveraging annotated corpora for accurate entity identification. However, the question arises: Can newer Large Language Models (LLMs) like GPT be utilized without the need for extensive annotation, thereby enabling direct entity extraction? In this study, we explore this issue, comparing the efficacy of fine-tuning techniques with prompting methods to elucidate the potential of GPT in the identification of medical entities within Spanish electronic health records (EHR). This study utilized a dataset of Spanish EHRs related to breast cancer and implemented both a traditional NER method using BERT, and a contemporary approach that combines few shot learning and integration of external knowledge, driven by LLMs using GPT, to structure the data. The analysis involved a comprehensive pipeline that included these methods. Key performance metrics, such as precision, recall, and F-score, were used to evaluate the effectiveness of each method. This comparative approach aimed to highlight the strengths and limitations of each method in the context of structuring Spanish EHRs efficiently and accurately.The comparative analysis undertaken in this article demonstrates that both the traditional BERT-based NER method and the few-shot LLM-driven approach, augmented with external knowledge, provide comparable levels of precision in metrics such as precision, recall, and F score when applied to Spanish EHR. Contrary to expectations, the LLM-driven approach, which necessitates minimal data annotation, performs on par with BERT’s capability to discern complex medical terminologies and contextual nuances within the EHRs. The results of this study highlight a notable advance in the field of NER for Spanish EHRs, with the few shot approach driven by LLM, enhanced by external knowledge, slightly edging out the traditional BERT-based method in overall effectiveness. GPT’s superiority in F-score and its minimal reliance on extensive data annotation underscore its potential in medical data processing.

CEPTNER: Contrastive learning Enhanced Prototypical network for Two-stage few-shot Named Entity Recognition

Few-shot Named Entity Recognition (NER) systems aim to classify unseen named entity types with limited labeled examples. Significant progress has been made in the use of large-scale pre-trained language models. However, boundary information and hidden relationships between entities beyond the sequence, which provide additional information and play crucial roles in few-shot NER, have received little attention in recent methods that achieve the state of the art. In this paper, we propose CEPTNER, a Contrastive learning Enhanced Prototypical network for Two-stage few-shot Named Entity Recognition, which leverages meta-learning and prototypical network to identify unseen entity types with limited labeled data. Concretely, we first detect candidate boundaries of entities in the stage of boundary detector, then we employ a prototypical network to filter false boundaries and type for the remaining spans in the stage of entity classifier. Additionally, we conduct entity-level contrastive learning to explore the internal relationship between entities that provides extra information while optimizing the prototypical network. CEPTNER is evaluated on two widely-used few-shot NER datasets and a few-shot slot tagging dataset: Few-NERD, CrossNER and SNIPS. Extensive experiments on various benchmarks show the superiority of CEPTNER over previous methods for few-shot NER.

Survey on Context Based Identification of Customer Name Variations using ML Techniques

Abstract: The demand for automated validation of new customer company accounts, with a specific emphasis on resolving name variations within large databases, is steadily increasing. This surge in demand is a direct response to the significant scale at which data for new customers is being input into CRM systems, necessitating manual validation and correction processes. The primary objective of this survey paper is to identify machine learning methods capable of addressing the challenge of resolving name variations in extensive databases. To achieve this goal, we propose a multi-step approach. Firstly, we employ an approximate string matching algorithm with high computational speed to identify matches with a very high percentage of accuracy. In cases where such matches are not found in the system, we utilize named entity recognition techniques to annotate company names and their suffixes, such as INC, PVT, LTD, CORP, and CO, which may appear in various forms. To resolve abbreviation disambiguity, we explore the application of machine learning algorithms, including the naïve Bayes classifier, decision trees, and Support Vector Machines. In this survey paper, we conclude by presenting potential approximate string matching algorithms, a named entity recognition method, and a model for resolving abbreviation disambiguity. Our review not only provides a comprehensive overview of the current state of research in this area but also highlights gaps in the existing knowledge, offering valuable insights for future research and practical application.

Span Graph Transformer for Document-Level Named Entity Recognition

Named Entity Recognition (NER), which aims to identify the span and category of entities within text, is a fundamental task in natural language processing. Recent NER approaches have featured pre-trained transformer-based models (e.g., BERT) as a crucial encoding component to achieve state-of-the-art performance. However, due to the length limit for input text, these models typically consider text at the sentence-level and cannot capture the long-range contextual dependency within a document. To address this issue, we propose a novel Span Graph Transformer (SGT) method for document-level NER, which constructs long-range contextual dependencies at both the token and span levels. Specifically, we first retrieve relevant contextual sentences in the document for each target sentence, and jointly encode them by BERT to capture token-level dependencies. Then, our proposed model extracts candidate spans from each sentence and integrates these spans into a document-level span graph, where nested spans within sentences and identical spans across sentences are connected. By leveraging the power of Graph Transformer and well-designed position encoding, our span graph can fully exploit span-level dependencies within the document. Extensive experiments on both resource-rich nested and flat NER datasets, as well as low-resource distantly supervised NER datasets, demonstrate that proposed SGT model achieves better performance than previous state-of-the-art models.

Robust Few-Shot Named Entity Recognition with Boundary Discrimination and Correlation Purification

Few-shot named entity recognition (NER) aims to recognize novel named entities in low-resource domains utilizing existing knowledge. However, the present few-shot NER models assume that the labeled data are all clean without noise or outliers, and there are few works focusing on the robustness of the cross-domain transfer learning ability to textual adversarial attacks in Few-shot NER. In this work, we comprehensively explore and assess the robustness of few-shot NER models under textual adversarial attack scenario, and found the vulnerability of existing few-shot NER models. Furthermore, we propose a robust two-stage few-shot NER method with Boundary Discrimination and Correlation Purification (BDCP). Specifically, in the span detection stage, the entity boundary discriminative module is introduced to provide a highly distinguishing boundary representation space to detect entity spans. In the entity typing stage, the correlations between entities and contexts are purified by minimizing the interference information and facilitating correlation generalization to alleviate the perturbations caused by textual adversarial attacks. In addition, we construct adversarial examples for few-shot NER based on public datasets Few-NERD and Cross-Dataset. Comprehensive evaluations on those two groups of few-shot NER datasets containing adversarial examples demonstrate the robustness and superiority of the proposed method.

A Robust Chinese Named Entity Recognition Method Based on Integrating Dual-Layer Features and CSBERT

In the rapidly evolving field of cybersecurity, the integration of multi-source, heterogeneous, and fragmented data into a coherent knowledge graph has garnered considerable attention. Such a graph elucidates semantic interconnections, thereby facilitating sophisticated analytical decision support. Central to the construction of a cybersecurity knowledge graph is Named Entity Recognition (NER), a critical technology that converts unstructured text into structured data. The efficacy of NER is pivotal, as it directly influences the integrity of the knowledge graph. The task of NER in cybersecurity, particularly within the Chinese linguistic context, presents distinct challenges. Chinese text lacks explicit space delimiters and features complex contextual dependencies, exacerbating the difficulty in discerning and categorizing named entities. These linguistic characteristics contribute to errors in word segmentation and semantic ambiguities, impeding NER accuracy. This paper introduces a novel NER methodology tailored for the Chinese cybersecurity corpus, termed CSBERT-IDCNN-BiLSTM-CRF. This approach harnesses Iterative Dilated Convolutional Neural Networks (IDCNN) for extracting local features, and Bi-directional Long Short-Term Memory networks (BiLSTM) for contextual understanding. It incorporates CSBERT, a pre-trained model adept at processing few-shot data, to derive input feature representations. The process culminates with Conditional Random Fields (CRF) for precise sequence labeling. To compensate for the scarcity of publicly accessible Chinese cybersecurity datasets, this paper synthesizes a bespoke dataset, authenticated by data from the China National Vulnerability Database, processed via the YEDDA annotation tool. Empirical analysis affirms that the proposed CSBERT-IDCNN-BiLSTM-CRF model surpasses existing Chinese NER frameworks, with an F1-score of 87.30% and a precision rate of 85.89%. This marks a significant advancement in the accurate identification of cybersecurity entities in Chinese text, reflecting the model’s robust capability to address the unique challenges presented by the language’s structural intricacies.

Transformer-based Named Entity Recognition for Clinical Cancer Drug Toxicity by Positive-unlabeled Learning and KL Regularizers

Background: With increasing rates of polypharmacy, the vigilant surveillance of clinical drug toxicity has emerged as an important concern. Named Entity Recognition (NER) stands as an indispensable undertaking, essential for the extraction of valuable insights regarding drug safety from the biomedical literature. In recent years, significant advancements have been achieved in the deep learning models on NER tasks. Nonetheless, the effectiveness of these NER techniques relies on the availability of substantial volumes of annotated data, which is labor-intensive and inefficient. background: With increasing rates of polypharmacy, clinical drug toxicity has been closely monitored. Named Entity Recognition (NER) is a vital task for extracting valuable drug safety information from biomedical literature. Recently, many deep learning models in biomedical domain have made great progress for NER, especially pre-trained language models. However, these NER methods require large amounts of high-quality manually annotated data with named entities, which is labor intensive and inefficient. Methods: This study introduces a novel approach that diverges from the conventional reliance on manually annotated data. It employs a transformer-based technique known as Positive-Unlabeled Learning (PULearning), which incorporates adaptive learning and is applied to the clinical cancer drug toxicity corpus. To improve the precision of prediction, we employ relative position embeddings within the transformer encoder. Additionally, we formulate a composite loss function that integrates two Kullback-Leibler (KL) regularizers to align with PULearning assumptions. The outcomes demonstrate that our approach attains the targeted performance for NER tasks, solely relying on unlabeled data and named entity dictionaries. objective: To improve the performance of prediction Conclusion: Our model achieves an overall NER performance with an F1 of 0.819. Specifically, it attains F1 of 0.841, 0.801 and 0.815 for DRUG, CANCER, and TOXI entities, respectively. A comprehensive analysis of the results validates the effectiveness of our approach in comparison to existing PULearning methods on biomedical NER tasks. Additionally, a visualization of the associations among three identified entities is provided, offering a valuable reference for querying their interrelationships. method: In this work, instead of relying on the manually labeled data, a transformer-based Positive-Unlabeled Learning (PULearning) is proposed with adaptive learning and applied on the clinical cancer drug toxicity corpus. To improve the precision of prediction, relative position embeddings are used in transformer encoder. And then, a mixed loss is designed with two Kullback-Leibler (KL) regularizers for PULearning assumptions. Through adaptive sampling, our approach meets the expected performance for NER task only using unlabeled data and named entity dictionaries. result: The overall NER performance of our model obtains 0.819 of F1-score, while 0.841, 0.801 and 0.815 of F1-score on DRUG, CANCER and TOXI, respectively. other: None

Template-Free Prompting for Few-Shot Named Entity Recognition via Semantic-Enhanced Contrastive Learning.

Prompt tuning has achieved great success in various sentence-level classification tasks by using elaborated label word mappings and prompt templates. However, for solving token-level classification tasks, e.g., named entity recognition (NER), previous research, which utilizes N-gram traversal for prompting all spans with all possible entity types, is time-consuming. To this end, we propose a novel prompt-based contrastive learning method for few-shot NER without template construction and label word mappings. First, we leverage external knowledge to initialize semantic anchors for each entity type. These anchors are simply appended with input sentence embeddings as template-free prompts (TFPs). Then, the prompts and sentence embeddings are in-context optimized with our proposed semantic-enhanced contrastive loss. Our proposed loss function enables contrastive learning in few-shot scenarios without requiring a significant number of negative samples. Moreover, it effectively addresses the issue of conventional contrastive learning, where negative instances with similar semantics are erroneously pushed apart in natural language processing (NLP)-related tasks. We examine our method in label extension (LE), domain-adaption (DA), and low-resource generalization evaluation tasks with six public datasets and different settings, achieving state-of-the-art (SOTA) results in most cases.

Context-Aware Attentive Multilevel Feature Fusion for Named Entity Recognition.

In the era of information explosion, named entity recognition (NER) has attracted widespread attention in the field of natural language processing, as it is fundamental to information extraction. Recently, methods of NER based on representation learning, e.g., character embedding and word embedding, have demonstrated promising recognition results. However, existing models only consider partial features derived from words or characters while failing to integrate semantic and syntactic information, e.g., capitalization, inter-word relations, keywords, and lexical phrases, from multilevel perspectives. Intuitively, multilevel features can be helpful when recognizing named entities from complex sentences. In this study, we propose a novel attentive multilevel feature fusion (AMFF) model for NER, which captures the multilevel features in the current context from various perspectives. It consists of four components to, respectively, capture the local character-level (CL), global character-level (CG), local word-level (WL), and global word-level (WG) features in the current context. In addition, we further define document-level features crafted from other sentences to enhance the representation learning of the current context. To this end, we introduce a novel context-aware attentive multilevel feature fusion (CAMFF) model based on AMFF, to fully leverage document-level features from all the previous inputs. The obtained multilevel features are then fused and fed into a bidirectional long short-term memory (BiLSTM)-conditional random field (CRF) network for the final sequence labeling. Extensive experiments on four benchmark datasets demonstrate that our proposed AMFF and CAMFF models outperform a set of state-of-the-art baseline methods and the features learned from multiple levels are complementary.

Применение метода распознавания именованных сущностей в сфере информационных технологий для анализа задач по разработке программного обеспечения

Explores the application of the Named Entity Recognition method to analyze software development tasks. It describes the main methods of labor-intensive analysis of IT companies, as well as NER technologies, libraries and frameworks such as SpaCy, DeepPavlov, Natasha and Pullenti, which can be used for extracting information about named entities from text data in Russian. This article offers useful resources and approaches for analyzing development tasks, which helps to improve planning, project management and reduce development time.

A Weakly Supervised Chinese Named Entity Recognition Method Combining First-Order Logic

A Weakly Supervised Chinese Named Entity Recognition Method Combining First-Order Logic

A Named Entity Recognition Method based on Knowledge Distillation and Efficient GlobalPointer for Chinese Medical Texts

A Named Entity Recognition Method based on Knowledge Distillation and Efficient GlobalPointer for Chinese Medical Texts

Chinese named entity recognition method based on multiscale feature fusion

Chinese named entity recognition method based on multiscale feature fusion

Chinese EMR Named Entity Recognition Using Fused Label Relations Based on Machine Reading Comprehension Framework.

Chinese electronic medical records (EMR) presents significant challenges for named entity recognition (NER) due to their specialized nature, unique language features, and diverse expressions. Traditionally, NER is treated as a sequence labeling task, where each token is assigned a label. Recent research has reframed NER within the machine reading comprehension (MRC) framework, extracting entities in a question-answer format, achieving state-of-the-art performance. However, these MRC-based methods have a significant limitation: they extract entities of various types independently, ignoring their interrelations. To address this, we introduce the Fusion Label Relations with MRC (FLR-MRC) model, which enhances the MRC model by implicitly capturing dependencies among entity types. FLR-MRC models interrelations between labels using graph attention networks, integrating these with textual data to identify entities. On the benchmark CMeEE and CCKS2017-CNER datasets, FLR-MRC achieves F1-scores of 0.6652 and 0.9101, respectively, outperforming existing clinical NER methods.

Class-Imbalanced-Aware Distantly Supervised Named Entity Recognition.

Distantly supervised named entity recognition (NER), which automatically learns NER models without manually labeling data, has gained much attention recently. In distantly supervised NER, positive unlabeled (PU) learning methods have achieved notable success. However, existing PU learning-based NER methods are unable to automatically handle the class imbalance and further depend on the estimation of the unknown class prior; thus, the class imbalance and imperfect estimation of the class prior degenerate the NER performance. To address these issues, this article proposes a novel PU learning method for distantly supervised NER. The proposed method can automatically handle the class imbalance and does not need to engage in class prior estimation, which enables the proposed methods to achieve the state-of-the-art performance. Extensive experiments support our theoretical analysis and validate the superiority of our method.

Named Entity Recognition in Electronic Health Records: A Methodological Review.

A substantial portion of the data contained in Electronic Health Records (EHR) is unstructured, often appearing as free text. This format restricts its potential utility in clinical decision-making. Named entity recognition (NER) methods address the challenge of extracting pertinent information from unstructured text. The aim of this study was to outline the current NER methods and trace their evolution from 2011 to 2022. We conducted a methodological literature review of NER methods, with a focus on distinguishing the classification models, the types of tagging systems, and the languages employed in various corpora. Several methods have been documented for automatically extracting relevant information from EHRs using natural language processing techniques such as NER and relation extraction (RE). These methods can automatically extract concepts, events, attributes, and other data, as well as the relationships between them. Most NER studies conducted thus far have utilized corpora in English or Chinese. Additionally, the bidirectional encoder representation from transformers using the BIO tagging system architecture is the most frequently reported classification scheme. We discovered a limited number of papers on the implementation of NER or RE tasks in EHRs within a specific clinical domain. EHRs play a pivotal role in gathering clinical information and could serve as the primary source for automated clinical decision support systems. However, the creation of new corpora from EHRs in specific clinical domains is essential to facilitate the swift development of NER and RE models applied to EHRs for use in clinical practice.

A System for Interviewing and Collecting Statements Based on Intent Classification and Named Entity Recognition Using Augmentation

In cases of child sexual abuse, interviewing and obtaining trustworthy statements from victims and witnesses is essential because their statements are the only evidence. It is crucial to ascertain objectively the credibility of the victim’s statements, which may vary based on the nature of the questions posed by the forensic interviewer. Therefore, interview skills that eliminate subjective opinions require a high level of training for forensic interviewers. To reduce high-risk subjective interviews, objectively analyzing statements is essential. Understanding the victim’s intent and named entity recognition (NER) in the statements is necessary to give the victim open-ended questions and memory recall. Therefore, the system provides an intent classification and NER method that follows the National Institute of Child Health and Human Development Investigative Interview Protocol, which outlines the collection of objective statements. Large language models such as BERT and KoBERT, along with data augmentation techniques, were proposed using a restricted training dataset of limited size to achieve effective intent classification and NER performance. Additionally, a system that can collect objective statements with the proposed model was developed and it was confirmed that it could assist statement analysts. The verification results showed that the model achieved average F1-scores of 95.5% and 97.8% for intent classification and NER, respectively, which improved the results of the limited data by 3.4% and 3.7%, respectively.

Research on Named Entity Recognition Method of Chinese Classics Under the Supervision of Domain Knowledge

The current dominant named entity recognition methods of Chinese classics are classified as data-driven methods, which are limited by the data quality. The domain knowledge is introduced in this paper to supervise the process of the named entity recognition, so as to solve the poor performance problem because of the low-quality data. The experiments on the Historical Records corpus show that compared with the domain knowledge unsupervised case, the average accuracy, recall rate, and F1 value have respectively improved by 2.76%, 2.70%, and 2.75% under the supervision of domain knowledge. Domain knowledge plays an important role in improving the performance of the named entity recognition methods of Chinese classics.

DdERT: Research on Named Entity Recognition for Mine Hoist Using a Chinese BERT Model

This study aims to solve the problem of named entity recognition of complex mechanical equipment faults, especially the problems of many professional terms, long sentences, fuzzy entity boundaries, entity nesting, and abbreviation ambiguity, in mine hoist fault text. Therefore, this study proposes a named entity recognition method based on domain dictionary embedding. The method first uses the fault domain knowledge of the mine hoist to construct a domain-specialized dictionary and generate a word vector of characteristic words. Secondly, the BERT pre-trained language model is used to obtain dynamic word vectors, and a dictionary adapter is loaded to obtain contextual domain lexical features to improve recognition accuracy. Finally, the conditional random field (CRF) is the model classifier to output the annotation sequence with the highest score. The experimental results show that this model achieves better than several baseline models and effectively improves the accuracy of fault named entity identification for mine hoists. The innovation of this study is the combination of domain dictionary embedding and a BERT pre-trained language model, which improves the accuracy and robustness of named entity recognition. Therefore, the results of this study have essential research significance for improving the accuracy of fault named entity identification of mine hoists and the construction of fault knowledge maps.

Named Entity Recognition in Power Marketing Domain Based on Whole Word Masking and Dual Feature Extraction

With the aim of solving the current problems of low utilization of entity features, multiple meanings of a word, and poor recognition of specialized terms in the Chinese power marketing domain named entity recognition (PMDNER), this study proposes a Chinese power marketing named entity recognition method based on whole word masking and joint extraction of dual features. Firstly, word vectorization of the electricity text data is performed using the RoBERTa pre-training model; then, it is fed into the constructed dual feature extraction neural network (DFENN) to acquire the local and global features of text in a parallel manner and fuse them. The output of the RoBERTa layer is used as the auxiliary classification layer, the output of the DFENN layer is used as the master classification layer, and the output of the two layers is dynamically combined through the attention mechanism to weight the outputs of the two layers so as to fuse new features, which are input into the conditional random field (CRF) layer to obtain the most reasonable label sequence. A focal loss function is used in the training process to alleviate the problem of uneven sample distribution. The experimental results show that the method achieved an F1 value of 88.58% on the constructed named entity recognition dataset in the power marketing domain, which is a significant improvement in performance compared with the existing methods.