Entity Recognition System Research Articles

Von OCR und HTR bis NER und LLM

Abstract The presence of the digital is closely linked to the expectation that work processes will be made easier and faster. Considering the barely recognizable increase in speed in editing projects, this article deals with the potentials and challenges of digital tools and processes which, at least in theory, have the potential to speed up editing processes by transferring tasks to the machine that would otherwise be carried out by humans. Using the examples of optical character recognition, handwritten text recognition, named entity recognition and LLM-based dialog systems (AI-based chatbots), the article examines what is associated with the integration of such digital methods and tools into the editing workflow and to what extent the work and transformation processes and, in particular, the roles of human editors and their specific tasks change.

Aviation-BERT-NER: Named Entity Recognition for Aviation Safety Reports

This work introduces Aviation-BERT-NER, a Named Entity Recognition (NER) system tailored for aviation safety reports, building on the Aviation-BERT base model developed at the Georgia Institute of Technology’s Aerospace Systems Design Laboratory. This system integrates aviation domain-specific data, including aircraft types, manufacturers, quantities, and aviation terminology, to identify named entities critical for aviation safety analysis. A key innovation of Aviation-BERT-NER is its template-based approach to fine-tuning, which utilizes structured datasets to generate synthetic training data that mirror the complexity of real-world aviation safety reports. This method significantly improves the model’s generalizability and adaptability, enabling rapid updates and customization to meet evolving domain-specific requirements. The development process involved careful data preparation, including the synthesis of entity types and the generation of labeled datasets through template filling. Testing on real-world narratives from the National Transportation Safety Board (NTSB) database highlighted Aviation-BERT-NER’s robustness, with a precision of 95.34%, recall of 94.62%, and F1 score of 94.78% when evaluated over 50 manually annotated (BIO tagged) paragraphs. This work addresses a critical gap in English language NER models for aviation safety, promising substantial improvements in the analysis and understanding of aviation safety reports.

Lifestyle factors in the biomedical literature: an ontology and comprehensive resources for named entity recognition.

Despite lifestyle factors (LSFs) being increasingly acknowledged in shaping individual health trajectories, particularly in chronic diseases, they have still not been systematically described in the biomedical literature. This is in part because no named entity recognition (NER) system exists, which can comprehensively detect all types of LSFs in text. The task is challenging due to their inherent diversity, lack of a comprehensive LSF classification for dictionary-based NER, and lack of a corpus for deep learning-based NER. We present a novel lifestyle factor ontology (LSFO), which we used to develop a dictionary-based system for recognition and normalization of LSFs. Additionally, we introduce a manually annotated corpus for LSFs (LSF200) suitable for training and evaluation of NER systems, and use it to train a transformer-based system. Evaluating the performance of both NER systems on the corpus revealed an F-score of 64% for the dictionary-based system and 76% for the transformer-based system. Large-scale application of these systems on PubMed abstracts and PMC Open Access articles identified over 300 million mentions of LSF in the biomedical literature. LSFO, the annotated LSF200 corpus, and the detected LSFs in PubMed and PMC-OA articles using both NER systems, are available under open licenses via the following GitHub repository: https://github.com/EsmaeilNourani/LSFO-expansion. This repository contains links to two associated GitHub repositories and a Zenodo project related to the study. LSFO is also available at BioPortal: https://bioportal.bioontology.org/ontologies/LSFO.

Exploring the Potential of Neural Machine Translation for Cross-Language Clinical Natural Language Processing (NLP) Resource Generation through Annotation Projection

Recent advancements in neural machine translation (NMT) offer promising potential for generating cross-language clinical natural language processing (NLP) resources. There is a pressing need to be able to foster the development of clinical NLP tools that extract key clinical entities in a comparable way for a multitude of medical application scenarios that are hindered by lack of multilingual annotated data. This study explores the efficacy of using NMT and annotation projection techniques with expert-in-the-loop validation to develop named entity recognition (NER) systems for an under-resourced target language (Catalan) by leveraging Spanish clinical corpora annotated by domain experts. We employed a state-of-the-art NMT system to translate three clinical case corpora. The translated annotations were then projected onto the target language texts and subsequently validated and corrected by clinical domain experts. The efficacy of the resulting NER systems was evaluated against manually annotated test sets in the target language. Our findings indicate that this approach not only facilitates the generation of high-quality training data for the target language (Catalan) but also demonstrates the potential to extend this methodology to other languages, thereby enhancing multilingual clinical NLP resource development. The generated corpora and components are publicly accessible, potentially providing a valuable resource for further research and application in multilingual clinical settings.

CEPTNER: Contrastive learning Enhanced Prototypical network for Two-stage few-shot Named Entity Recognition

Few-shot Named Entity Recognition (NER) systems aim to classify unseen named entity types with limited labeled examples. Significant progress has been made in the use of large-scale pre-trained language models. However, boundary information and hidden relationships between entities beyond the sequence, which provide additional information and play crucial roles in few-shot NER, have received little attention in recent methods that achieve the state of the art. In this paper, we propose CEPTNER, a Contrastive learning Enhanced Prototypical network for Two-stage few-shot Named Entity Recognition, which leverages meta-learning and prototypical network to identify unseen entity types with limited labeled data. Concretely, we first detect candidate boundaries of entities in the stage of boundary detector, then we employ a prototypical network to filter false boundaries and type for the remaining spans in the stage of entity classifier. Additionally, we conduct entity-level contrastive learning to explore the internal relationship between entities that provides extra information while optimizing the prototypical network. CEPTNER is evaluated on two widely-used few-shot NER datasets and a few-shot slot tagging dataset: Few-NERD, CrossNER and SNIPS. Extensive experiments on various benchmarks show the superiority of CEPTNER over previous methods for few-shot NER.

BERT-based tourism named entity recognition: making use of social media for travel recommendations.

Social media has become a massive encyclopedia of almost anything due to its content richness. People tell stories, write comments and feedback, and share knowledge through social media. The information available on social media enables 'clueless' travelers to get quick travel recommendations in the tourism sector. Through a simple query, such as typing 'places to visit in Bali', travelers can get many blog articles to help them decide which places of interest to visit. However, doing this reading task without a helper can be overwhelming. To overcome this problem, we developed Bidirectional Encoder Representations from Transformers (BERT)-based tourism named entity recognition system, which is used to highlight tourist destination places in the query result. BERT is a state-of-the-art machine learning framework for natural language processing that can give a decent performance in various settings and cases. Our developed tourism named entity recognition (NER) model specifies three different tourist destinations: heritage, natural, and purposefully built (man-made or artificial). The dataset is taken from various tourism-related community articles and posts. The model achieved an average F1-score of 0.80 and has been implemented into a traveling destination recommendation system. By using this system, travelers can get quick recommendations based on the popularity of places visited in the query frame. Based on the survey that we conducted to target respondents who have never visited and have no or limited knowledge about tourist attractions in some example cities, their average interest level from the recommendation results is higher than four on a scale of 1 to 5. Thus, it can be considered a good recommendation. Furthermore, the NER model performance is comparable to another related research.

Named Entity Recognition and Classification in Historical Documents: A Survey

After decades of massive digitisation, an unprecedented number of historical documents are available in digital format, along with their machine-readable texts. While this represents a major step forward with respect to preservation and accessibility, it also opens up new opportunities in terms of content mining and the next fundamental challenge is to develop appropriate technologies to efficiently search, retrieve, and explore information from this ‘big data of the past’. Among semantic indexing opportunities, the recognition and classification of named entities are in great demand among humanities scholars. Yet, named entity recognition (NER) systems are heavily challenged with diverse, historical, and noisy inputs. In this survey, we present the array of challenges posed by historical documents to NER, inventory existing resources, describe the main approaches deployed so far, and identify key priorities for future developments.

QUARE: towards a question-answering model for requirements elicitation

Requirements elicitation is a stakeholder-centered approach; therefore, natural language remains an effective way of documenting and validating requirements. As the scope of the software domain grows, software analysts process a higher number of requirements documents, generating delays and errors while characterizing the software domain. Natural language processing is key in such a process, allowing software analysts for speeding up the requirements elicitation process and mitigating the impact of the ambiguity and misinterpretations coming from natural-language-based requirements documents. However, natural-language-processing-based proposals for requirements elicitation are mainly focused on specific domains and still fail for understanding several requirements writing styles. In this paper, we present QUARE, a question-answering model for requirements elicitation. The QUARE model comprises a meta-ontology for requirements elicitation, easing the generation of requirements-elicitation-related questions and the initial structuration of any software domain. In addition, the QUARE model includes a named entity recognition and relation extraction system focused on requirements elicitation, allowing software analysts for processing several requirements writing styles. Although software analysts address a software domain at a time, they use the same kind of questions for identifying and characterizing requirements abstractions such as actors, concepts, and actions from a software domain. Such a process may be framed into the QUARE model workflow. We validate our proposal by using an experimental process including real-world requirements documents coming from several software domains and requirements writing styles. The QUARE model is a novel proposal aimed at supporting software analysts in the requirements elicitation process.

Reinforcement learning based distantly supervised biomedical named entity recognition

Data used by current Biomedical named entity recognition (BioNER) systems has mostly been manually labelled for supervision. However, it might be difficult to find large amounts of annotated data, especially in fields with a high level of specialization, such as biomedical, bioinformatics, and so on. When dictionaries and ontologies are available, which are domain-specific knowledge resources, automatically tagged distantly supervised biomedical training data can be developed. However, any such distantly supervised NER result is normally noisy. The prevalence of false positives and false negatives with this type of autonomously generated data is the main problem that directly affects efficiency. This research investigates distant supervision to detect false positive occurrences in BioNER task. A reinforcement learning technique is employed that is modelled as a graphical processing unit (GPU) accelerated Markov decision process (MDP) with a neural network policy. To deal with false negative cases, we employ a partial annotation conditional random field (CRF) technique. Results on two benchmark datasets show a cutting-edge methodology that can enhance the functionality of the neural NER system. It goes on to show how the proposed approach cuts down on human annotated data for BioNER tasks in Natural Language Processing (NLP).

Automatic extraction of social determinants of health from medical notes of chronic lower back pain patients.

We applied natural language processing and inference methods to extract social determinants of health (SDoH) information from clinical notes of patients with chronic low back pain (cLBP) to enhance future analyses of the associations between SDoH disparities and cLBP outcomes. Clinical notes for patients with cLBP were annotated for 7 SDoH domains, as well as depression, anxiety, and pain scores, resulting in 626 notes with at least one annotated entity for 364 patients. We used a 2-tier taxonomy with these 10 first-level classes (domains) and 52 second-level classes. We developed and validated named entity recognition (NER) systems based on both rule-based and machine learning approaches and validated an entailment model. Annotators achieved a high interrater agreement (Cohen's kappa of 95.3% at document level). A rule-based system (cTAKES), RoBERTa NER, and a hybrid model (combining rules and logistic regression) achieved performance of F1 = 47.1%, 84.4%, and 80.3%, respectively, for first-level classes. While the hybrid model had a lower F1 performance, it matched or outperformed RoBERTa NER model in terms of recall and had lower computational requirements. Applying an untuned RoBERTa entailment model, we detected many challenging wordings missed by NER systems. Still, the entailment model may be sensitive to hypothesis wording. This study developed a corpus of annotated clinical notes covering a broad spectrum of SDoH classes. This corpus provides a basis for training machine learning models and serves as a benchmark for predictive models for NER for SDoH and knowledge extraction from clinical texts.

Combination of Loss-based Active Learning and Semi-supervised Learning for Recognizing Entities in Chinese Electronic Medical Records

The recognition of entities in an electronic medical record (EMR) is especially important to downstream tasks, such as clinical entity normalization and medical dialogue understanding. However, in the medical professional field, training a high-quality named entity recognition system always requires large-scale annotated datasets, which are highly expensive to obtain. In this article, to lower the cost of data annotation and maximizing the use of unlabeled data, we propose a hybrid approach to recognizing the entities in Chinese electronic medical record, which is in combination of loss-based active learning and semi-supervised learning. Specifically, we adopted a dynamic balance strategy to dynamically balance the minimum loss predicted by a named entity recognition decoder and a loss prediction module at different stages in the process. Experimental results demonstrated our proposed framework’s effectiveness and efficiency, achieving higher performances than existing approaches on Chinese EMR entity recognition datasets under limited labeling resources.

Extracting social determinants of health events with transformer-based multitask, multilabel named entity recognition.

Social determinants of health (SDOH) are nonclinical, socioeconomic conditions that influence patient health and quality of life. Identifying SDOH may help clinicians target interventions. However, SDOH are more frequently available in narrative notes compared to structured electronic health records. The 2022 n2c2 Track 2 competition released clinical notes annotated for SDOH to promote development of NLP systems for extracting SDOH. We developed a system addressing 3 limitations in state-of-the-art SDOH extraction: the inability to identify multiple SDOH events of the same type per sentence, overlapping SDOH attributes within text spans, and SDOH spanning multiple sentences. We developed and evaluated a 2-stage architecture. In stage 1, we trained a BioClinical-BERT-based named entity recognition system to extract SDOH event triggers, that is, text spans indicating substance use, employment, or living status. In stage 2, we trained a multitask, multilabel NER to extract arguments (eg, alcohol "type") for events extracted in stage 1. Evaluation was performed across 3 subtasks differing by provenance of training and validation data using precision, recall, and F1 scores. When trained and validated on data from the same site, we achieved 0.87 precision, 0.89 recall, and 0.88 F1. Across all subtasks, we ranked between second and fourth place in the competition and always within 0.02 F1 from first. Our 2-stage, deep-learning-based NLP system effectively extracted SDOH events from clinical notes. This was achieved with a novel classification framework that leveraged simpler architectures compared to state-of-the-art systems. Improved SDOH extraction may help clinicians improve health outcomes.

Noise Reduction Learning Based on XLNet-CRF for Biomedical Named Entity Recognition.

In recent years, Biomedical Named Entity Recognition (BioNER) systems have mainly been based on deep neural networks, which are used to extract information from the rapidly expanding biomedical literature. Long-distance context autoencoding language models based on transformers have recently been employed for BioNER with great success. However, noise interference exists in the process of pre-training and fine-tuning, and there is no effective decoder for label dependency. Current models have many aspects in need of improvement for better performance. We propose two kinds of noise reduction models, Shared Labels and Dynamic Splicing, based on XLNet encoding which is a permutation language pre-training model and decoding by Conditional Random Field (CRF). By testing 15 biomedical named entity recognition datasets, the two models improved the average F1-score by 1.504 and 1.48, respectively, and state-of-the-art performance was achieved on 7 of them. Further analysis proves the effectiveness of the two models and the improvement of the recognition effect of CRF, and suggests the applicable scope of the models according to different data characteristics.

An Emphatic Attempt with Cognizance of the Marathi Language for Named Entity Recognition

Recent developments in the field of artificial intelligence has led to renewed interest in natural language processing. Named entity recognition (NER) is a classic problem in natural language processing. Investigating named entities is a continuing concern within information extraction. Names of people, organizations, locations, events, etc. are named entities (NEs). Although extensive research has been carried out on NER, limited study exists that explores NER in the Marathi language. The paper aims to provide a conceptual theoretical framework based on the gazetteer matching technique for named entity recognition system development for the Marathi language. A combination of rules and regular expressions is considered for temporal and numerical pattern recognition. The techniques presented in this paper are systematic, clear, and effective for morphologically rich language processing tool development. The system described in the paper reports a satisfactory performance with 62.64% NE identification and 72.27% NE classification accuracy.

ANEC: An Amharic Named Entity Corpus and Transformer Based Recognizer

Named Entity Recognition is an information extraction task that serves as a pre-processing step for other natural language processing tasks, such as machine translation, information retrieval, and question answering. Named entity recognition enables the identification of proper names as well as temporal and numeric expressions in an open domain text. For Semitic languages such as Arabic, Amharic, and Hebrew, the named entity recognition task is more challenging due to the heavily inflected structure of these languages. In this study, we annotate a new comparatively large Amharic named entity recognition dataset and make it publicly available. Using this new dataset, we build multiple Amharic named entity recognition systems based on recent deep learning approaches including transfer learning (RoBERTa), and bidirectional long short-term memory coupled with a conditional random fields layer. By applying the Synthetic Minority Over-sampling Technique to mitigate the imbalanced classification problem, our best performing RoBERTa based named entity recognition system achieves an f1-score of 93%, which is the new state-of-the-art result for Amharic named entity recognition.

Open Named Entity Modeling From Embedding Distribution

In this paper, we report our discovery on named entity distribution in a general word embedding space, which helps an open definition on multilingual named entity definition rather than previous closed and constraint definition on named entities through a named entity dictionary, which is usually derived from human labor and replies on schedule update. Our initial visualization of monolingual word embeddings indicates named entities tend to gather together despite of named entity types and language difference, which enable us to model all named entities using a specific geometric structure inside embedding space, namely, the named entity hypersphere. For monolingual cases, the proposed named entity model gives an open description of diverse named entity types and different languages. For cross-lingual cases, mapping the proposed named entity model provides a novel way to build a named entity dataset for resource-poor languages. At last, the proposed named entity model may be shown as a handy clue to enhance state-of-the-art named entity recognition systems generally.

Reconhecimento Automático Multilingue de Entidades Mencionadas em Diversos Domínios, para Efeitos de Anonimização de Tradução Automática

The following article describes the research developed at Unbabel, a Portuguese Machine-Translation start-up, that combines Machine Translation (MT) with human post-edition with a focus on customer service content. With the work carried out within a real multilingual AI powered, human-refined, MT industry, we aim to contribute to furthering MT quality and good-practices, by exposing the importance of having continuously in development, robust Named Entity Recognition systems for General Data Protection Regulation (GDPR) compliance. We will report three different experiments, resulting from a shared work with Unbabel´s linguists and Unbabel´s Artificial Intelligence (AI) engineering team, matured over a year. The first experiment focused on developing a methodology for the identification and annotation of domain-specific Named Entities (NEs) for the Food-Industry. The devised methodology allows the construction of gold standards for building domain specific NER systems and can be applied for a myriad of different domains. With the implementation of the designed method, we were able to identify the following domain-specific NEs set: Restaurant Names; Restaurant Chains; Dishes; Beverage, Ingredients. The second and third experiments explored the possibilities of constructing, in a semi-automatically way, multilingual NER gold standards for different domains and language pairs, using aligners that project Named Entities across a parallel corpus. Both experiments made it possible to benchmark four different open-source aligners (SimAlign; Fastalign; AwesomeAlign; Eflomal), allowing to identify the one with better performance and, simultaneously, validate the aforementioned approach. This work should be taken as a statement of multidisciplinary, proving and validating the much-needed articulation between different scientific fields that compose and characterize the area of Natural Language Processing (NLP).

Deep Learning-Based Named Entity Recognition System Using Hybrid Embedding

Retrieval of meaningful information out of voluminous data available on the internet is a big challenge these days. Named entity recognition system deals with this challenge efficiently and achieves promising results in accessing information of interest in several NLP applications. The present study proposes a deep learning-based named entity recognition system using hybrid embedding which is the combination of fasttext and bidirectional LSTM based character embedding. These embeddings capture the contextual, syntactic and semantic properties of the text which improves the cognitive power of deep learning methods. We have performed different experiments with important variants of recurrent neural network (RNN) namely long short-term memory network (LSTM) and bidirectional LSTM as well as gated recurrent unit (GRU) and bidirectional GRU on manually annotated Punjabi dataset and annotated Hindi dataset collected from international joint conference on natural language processing (IJCNLP-08) website. We have also explored different word embeddings and character embeddings for named entity recognition task. Out of all the experiments, the bidirectional GRU model using hybrid embedding has outperformed with precision, recall, and f-score values of 84%, 83%, and 83.50% respectively for Punjabi named entity recognition and 75%, 77%, 75.99% respectively for Hindi named entity recognition.

CL-NERIL: A Cross-Lingual Model for NER in Indian Languages (Student Abstract)

Developing Named Entity Recognition (NER) systems for Indian languages has been a long-standing challenge, mainly owing to the requirement of a large amount of annotated clean training instances. This paper proposes an end-to-end framework for NER for Indian languages in a low-resource setting by exploiting parallel corpora of English and Indian languages and an English NER dataset. The proposed framework includes an annotation projection method that combines word alignment score and NER tag prediction confidence score on source language (English) data to generate weakly labeled data in a target Indian language. We employ a variant of the Teacher-Student model and optimize it jointly on the pseudo labels of the Teacher model and predictions on the generated weakly labeled data. We also present manually annotated test sets for three Indian languages: Hindi, Bengali, and Gujarati. We evaluate the performance of the proposed framework on the test sets of the three Indian languages. Empirical results show a minimum 10% performance improvement compared to the zero-shot transfer learning model on all languages. This indicates that weakly labeled data generated using the proposed annotation projection method in target Indian languages can complement well-annotated source language data to enhance performance. Our code is publicly available at https://github.com/aksh555/CL-NERIL.

A Unified Model Using Distantly Supervised Data and Cross-Domain Data in NER.

Named entity recognition (NER) systems are often realized by supervised methods that require large hand-annotated data. When the hand-annotated data is limited, distantly supervised (DS) data and cross-domain (CD) data are usually used separately to improve the performance. The distantly supervised data can provide in-domain dictionary information, and the hand-annotated cross-domain information can be provided by cross-domain data. These two types of information are complemental. However, there are two problems required to be solved before using directly. First, the distantly supervised data may contain a lot of noise. Second, directly using cross-domain data may degrade performance due to the distribution mismatching problem. In this paper, we propose a unified model named PARE (PArtial learning and REinforcement learning). The PARE model can simultaneously use distantly supervised data and cross-domain data as external data. The model uses the partial learning method with a new label strategy to better handle the noise in distantly supervised data. The reinforcement learning method is used to alleviate the distribution mismatching problem in cross-domain data. Experiments in three datasets show that our model outperforms other baseline models. Besides, our model can be used in the situation where no hand-annotated in-domain data is provided.