Named Entity Recognition Approach Research Articles

FE-CFNER: Feature Enhancement-based approach for Chinese Few-shot Named Entity Recognition

Although significant progress has been made in Chinese Named Entity Recognition (NER) methods based on deep learning, their performance often falls short in few-shot scenarios. Feature enhancement is considered a promising approach to address the issue of Chinese few-shot NER. However, traditional feature fusion methods tend to lead to the loss of important information and the integration of irrelevant information. Despite the benefits of incorporating BERT for improving entity recognition, its performance is limited when training data is insufficient. To tackle these challenges, this paper proposes a Feature Enhancement-based approach for Chinese Few-shot NER called FE-CFNER. FE-CFNER designs a double cross neural network to minimize information loss through the interaction of feature cross twice. Additionally, adaptive weights and a top-k mechanism are introduced to sparsify attention distributions, enabling the model to prioritize important information related to entities while excluding irrelevant information. To further enhance the quality of BERT embeddings, FE-CFNER employs a contrastive template for contrastive learning pre-training of BERT, enhancing BERT’s semantic understanding capability. We evaluate the proposed method on four sampled Chinese NER datasets: Weibo, Resume, Taobao, and Youku. Experimental results validate the effectiveness and superiority of FE-CFNER in Chinese few-shot NER tasks.

Comparing NER Approaches on French Clinical Text, with Easy-to-Reuse Pipelines.

The task of Named Entity Recognition (NER) is central for leveraging the content of clinical texts in observational studies. Indeed, texts contain a large part of the information available in Electronic Health Records (EHRs). However, clinical texts are highly heterogeneous between healthcare services and institutions, between countries and languages, making it hard to predict how existing tools may perform on a particular corpus. We compared four NER approaches on three French corpora and share our benchmarking pipeline in an open and easy-to-reuse manner, using the medkit Python library. We include in our pipelines fine-tuning operations with either one or several of the considered corpora. Our results illustrate the expected superiority of language models over a dictionary-based approach, and question the necessity of refining models already trained on biomedical texts. Beyond benchmarking, we believe sharing reusable and customizable pipelines for comparing fast-evolving Natural Language Processing (NLP) tools is a valuable contribution, since clinical texts themselves can hardly be shared for privacy concerns.

A deep learning approach for Named Entity Recognition in Urdu language.

Named Entity Recognition (NER) is a natural language processing task that has been widely explored for different languages in the recent decade but is still an under-researched area for the Urdu language due to its rich morphology and language complexities. Existing state-of-the-art studies on Urdu NER use various deep-learning approaches through automatic feature selection using word embeddings. This paper presents a deep learning approach for Urdu NER that harnesses FastText and Floret word embeddings to capture the contextual information of words by considering the surrounding context of words for improved feature extraction. The pre-trained FastText and Floret word embeddings are publicly available for Urdu language which are utilized to generate feature vectors of four benchmark Urdu language datasets. These features are then used as input to train various combinations of Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), Gated Recurrent Unit (GRU), CRF, and deep learning models. The results show that our proposed approach significantly outperforms existing state-of-the-art studies on Urdu NER, achieving an F-score of up to 0.98 when using BiLSTM+GRU with Floret embeddings. Error analysis shows a low classification error rate ranging from 1.24% to 3.63% across various datasets showing the robustness of the proposed approach. The performance comparison shows that the proposed approach significantly outperforms similar existing studies.

Span Graph Transformer for Document-Level Named Entity Recognition

Named Entity Recognition (NER), which aims to identify the span and category of entities within text, is a fundamental task in natural language processing. Recent NER approaches have featured pre-trained transformer-based models (e.g., BERT) as a crucial encoding component to achieve state-of-the-art performance. However, due to the length limit for input text, these models typically consider text at the sentence-level and cannot capture the long-range contextual dependency within a document. To address this issue, we propose a novel Span Graph Transformer (SGT) method for document-level NER, which constructs long-range contextual dependencies at both the token and span levels. Specifically, we first retrieve relevant contextual sentences in the document for each target sentence, and jointly encode them by BERT to capture token-level dependencies. Then, our proposed model extracts candidate spans from each sentence and integrates these spans into a document-level span graph, where nested spans within sentences and identical spans across sentences are connected. By leveraging the power of Graph Transformer and well-designed position encoding, our span graph can fully exploit span-level dependencies within the document. Extensive experiments on both resource-rich nested and flat NER datasets, as well as low-resource distantly supervised NER datasets, demonstrate that proposed SGT model achieves better performance than previous state-of-the-art models.

RSRNeT: a novel multi-modal network framework for named entity recognition and relation extraction

Named entity recognition (NER) and relation extraction (RE) are two important technologies employed in knowledge extraction for constructing knowledge graphs. Uni-modal NER and RE approaches solely rely on text information for knowledge extraction, leading to various limitations, such as suboptimal performance and low efficiency in recognizing polysemous words. With the development of multi-modal learning, multi-modal named entity recognition (MNER) and multi-modal relation extraction (MRE) have been introduced to improve recognition performance. However, existing MNER and MRE methods often encounter reduced efficiency when the text includes unrelated images. To address this problem, we propose a novel multi-modal network framework for named entity recognition and relation extraction called RSRNeT. In RSRNeT, we focus on extracting visual features more fully and designing a multi-scale visual feature extraction module based on ResNeSt network. On the other hand, we also emphasize fusing multi-modal features more comprehensively while minimizing interference from irrelevant images. To address this issue, we propose a multi-modal feature fusing module based on RoBERTa network. These two modules enable us to learn superior visual and textual representations, reducing errors caused by irrelevant images. Our approach has undergone extensive evaluation and comparison with various baseline models on MNER and MRE tasks. Experimental results show that our method achieves state-of-the-art performance in recall and F1 score on three public datasets: Twitter2015, Twitter2017 and MNRE.

A Transfer Learning-Based Pairwise Information Extraction Framework Using BERT and Korean-Language Modification Relationships

Most named entity recognition approaches employing BERT-based transfer learning focus solely on extracting independent and simple tags, neglecting the sequence and dependency features inherent in the named-entity tags. Consequently, these basic BERT-based methods fall short in domains requiring the extraction of more intricate information, such as the detailed characteristics of products, services, and places from user reviews. In this paper, we introduce an end-to-end information extraction framework comprising three key components: (1) a tagging scheme that effectively represents detailed characteristics; (2) a BERT-based transfer learning model designed for extracting named-entity tags, utilizing both general linguistic features learned from a large corpus and the sequence and symmetric-dependency features of the named-entity tags; and (3) a pairwise information extraction algorithm that pairs features with their corresponding symmetric modifying words to extract detailed information.

A Named Entity Recognition Approach for Electronic Medical Records Using BERT Semantic Enhancement and BiLSTM

Aiming at the problems of missing local context features, single word vector representation, and low entity recognition accuracy, a method for e-medical recording with named entity recognition, which is based on BERT and model fusion, is proposed. First, with the model of BERT for pre-training, the preceding and following contextual information is fused for the enhancement of word semantic representation and alleviation of the problem of polysemy; second, the network of bi-directional long-short term memory is for obtaining the sequence feature matrix, generation of optimal sequence in global sense achieved through the conditional random field model; finally, data enhancement is used to alleviate the class imbalance and improve the model ability in generalization. Results of the experiments find model proposal measured by F1 on CCKS21 data set reaches 0.8548, which is 0.51% and 0.08% higher than models with ID-CNNs-CRF and multi-task RNN. This demonstrates the excellent performance of the method proposed in this paper in improving named entity recognition.

Europe PMC annotated full-text corpus for gene/proteins, diseases and organisms

Named entity recognition (NER) is a widely used text-mining and natural language processing (NLP) subtask. In recent years, deep learning methods have superseded traditional dictionary- and rule-based NER approaches. A high-quality dataset is essential to fully leverage recent deep learning advancements. While several gold-standard corpora for biomedical entities in abstracts exist, only a few are based on full-text research articles. The Europe PMC literature database routinely annotates Gene/Proteins, Diseases, and Organisms entities. To transition this pipeline from a dictionary-based to a machine learning-based approach, we have developed a human-annotated full-text corpus for these entities, comprising 300 full-text open-access research articles. Over 72,000 mentions of biomedical concepts have been identified within approximately 114,000 sentences. This article describes the corpus and details how to access and reuse this open community resource.

Geo-knowledge-guided GPT models improve the extraction of location descriptions from disaster-related social media messages

Social media messages posted by people during natural disasters often contain important location descriptions, such as the locations of victims. Recent research has shown that many of these location descriptions go beyond simple place names, such as city names and street names, and are difficult to extract using typical named entity recognition (NER) tools. While advanced machine learning models could be trained, they require large labeled training datasets that can be time-consuming and labor-intensive to create. In this work, we propose a method that fuses geo-knowledge of location descriptions and a Generative Pre-trained Transformer (GPT) model, such as ChatGPT and GPT-4. The result is a geo-knowledge-guided GPT model that can accurately extract location descriptions from disaster-related social media messages. Also, only 22 training examples encoding geo-knowledge are used in our method. We conduct experiments to compare this method with nine alternative approaches on a dataset of tweets from Hurricane Harvey. Our method demonstrates an over 40% improvement over typically used NER approaches. The experiment results also show that geo-knowledge is indispensable for guiding the behavior of GPT models. The extracted location descriptions can help disaster responders reach victims more quickly and may even save lives.

MCA-NER: Multi-Contextualized Adversarial-Based Attentional Deep Neural Network for Named Entity Recognition

Multi-contextualized representations learning is vital for named entity recognition (NER), which is a fundamental task for effectively extracting structured information from unstructured text, and forming knowledge bases. This task is particularly challenging when dealing with Chinese text given the absence of evident word boundaries. Chinese word segmentation (CWS) can be leveraged to recognize word boundaries, but named entities often encompass multiple segmented words, making it crucial to use boundary information to correctly recognize and distinguish the relationships between these words. In this paper, we propose MCA-NER, a multi-contextualized adversarial-based attentional deep learning approach for Chinese NER, which combines CWS and part-of-speech (POS) tagging information with the classic BiLSTM-CRF NER model, using adversarial multi-task learning. The model incorporates several self-attention components for adversarial and multi-task learning, effectively synthesizing task-specific and common information attribution while improving performance across all three tasks. Experimental results on the three datasets provide compelling evidence that supports the effectiveness and performance of our model.

Automated Identification of Active Players for International Construction Market Entry Using Natural Language Processing

Studies on the international construction market have been limited to expanding the scope of academics and practices because of data accessibility and timeliness. With the recent advancement of natural language processing (NLP) technologies, it becomes possible to extract on-time information from online news articles automatically. As a point of departure for developing a text-based information extraction model, this study aims to develop a named entity recognition (NER) model that automatically detects active players from news articles in the international construction industry. NER is an essential subtask of information extraction that automatically identifies key elements and classifies them into predefined categories. The proposed model detects owners, contractors, and consultants from news articles. The performance of the experiment was measured by a micro average F1 score of 85.8% with precision and recall values of 84.2% and 87.4%, respectively. This study contributes to investigating international market participants in a timely way with enhanced data accessibility. Therefore, the following studies will enlarge the NER approach to recognize “Who contacts whom,” “Who claims whom,” and “What delays what projects,” which will lead to extracting more valuable information automatically in the future.

A framework for multi-faceted content analysis of social media chatter regarding non-medical use of prescription medications

BackgroundSubstance use, including the non-medical use of prescription medications, is a global health problem resulting in hundreds of thousands of overdose deaths and other health problems. Social media has emerged as a potent source of information for studying substance use-related behaviours and their consequences. Mining large-scale social media data on the topic requires the development of natural language processing (NLP) and machine learning frameworks customized for this problem. Our objective in this research is to develop a framework for conducting a content analysis of Twitter chatter about the non-medical use of a set of prescription medications.MethodsWe collected Twitter data for four medications—fentanyl and morphine (opioids), alprazolam (benzodiazepine), and Adderall® (stimulant), and identified posts that indicated non-medical use using an automatic machine learning classifier. In our NLP framework, we applied supervised named entity recognition (NER) to identify other substances mentioned, symptoms, and adverse events. We applied unsupervised topic modelling to identify latent topics associated with the chatter for each medication.ResultsThe quantitative analysis demonstrated the performance of the proposed NER approach in identifying substance-related entities from data with a high degree of accuracy compared to the baseline methods. The performance evaluation of the topic modelling was also notable. The qualitative analysis revealed knowledge about the use, non-medical use, and side effects of these medications in individuals and communities.ConclusionsNLP-based analyses of Twitter chatter associated with prescription medications belonging to different categories provide multi-faceted insights about their use and consequences. Our developed framework can be applied to chatter about other substances. Further research can validate the predictive value of this information on the prevention, assessment, and management of these disorders.

A Chinese nested named entity recognition approach using sequence labeling

PurposeThis study aims to introduce an innovative approach that uses a decoder with multiple layers to accurately identify Chinese nested entities across various nesting depths. To address potential human intervention, an advanced optimization algorithm is used to fine-tune the decoder based on the depth of nested entities present in the data set. With this approach, this study achieves remarkable performance in recognizing Chinese nested entities.Design/methodology/approachThis study provides a framework for Chinese nested named entity recognition (NER) based on sequence labeling methods. Similar to existing approaches, the framework uses an advanced pre-training model as the backbone to extract semantic features from the text. Then a decoder comprising multiple conditional random field (CRF) algorithms is used to learn the associations between granularity labels. To minimize the need for manual intervention, the Jaya algorithm is used to optimize the number of CRF layers. Experimental results validate the effectiveness of the proposed approach, demonstrating its superior performance on both Chinese nested NER and flat NER tasks.FindingsThe experimental findings illustrate that the proposed methodology can achieve a remarkable 4.32% advancement in nested NER performance on the People’s Daily corpus compared to existing models.Originality/valueThis study explores a Chinese NER methodology based on the sequence labeling ideology for recognizing sophisticated Chinese nested entities with remarkable accuracy.

Named Entity Recognition in Russian Using Multi-Task LSTM-CRF

Named entity recognition (NER) is aimed at obtaining the important information from the unstructured data presented in the form of natural language texts. In this work, we investigate the efficiency of modern multi-task NER approaches on Russian language corpora by employing several different NER datasets and a dataset of part-of-speech (POS) tags. We apply a state-of-the-art neural architecture based on bidirectional LSTMs and conditional random fields. Convolutional neural networks were utilized to learn character-level features. We carry out extensive experimental evaluation over three standard datasets of news articles written in Russian. The proposed multi-task model achieve state-of-the-art results with an F1 score of 88.04% on Gareev’s dataset and an F1 score of 99.49% on Person-1000 dataset.

Using Data Augmentation and Bidirectional Encoder Representations from Transformers for Improving Punjabi Named Entity Recognition

Named entity recognition (NER) is a task of proper noun identification from natural language text and classification into various types such as location, person, and organization. Due to NER's applications in different natural language processing (NLP) tasks, numerous NER approaches and benchmark datasets have been proposed. However, developing NER techniques for low-resource languages is still limited due to the absence of substantial training datasets. Punjabi is a classic example of low resource language. Although various researchers have worked on Punjabi, they focused on the Gurmukhi script. To overcome the challenges in developing NER for the Shahmukhi script, we present an improved technique for Punjabi NER for the Shahmukhi script in this paper. We firstly extend the existing dataset by adding new NER classes by leveraging a novel Pool of Words data augmentation strategy. Our extended dataset has 11,31,509 tokens and 1,25,789 labeled entities with more named entities (NEs) than the older dataset. In the next step, we fine-tuned a transformer model known as Bidirectional Encoder Representations from Transformers (BERT) for the NER task. We performed experiments using the proposed approach on a new and older dataset version, showing that our method achieved competitive results.

Indonesian news classification application with named entity recognition approach

Nowadays, many netizens search for news via search engines with countless amounts of information, so it is increasingly difficult to determine when the number of news articles that appear changes very quickly and dynamically. Thus, it is necessary to process the extraction of news information to display the core information of the news. Problems arise, especially in Indonesian, which has a structure of various noun phrase entities with shallow parsing or grammatical induction. Named Entity Recognition (NER) has the opportunity to overcome this because it can extract news entities in depth, starting from proper nouns in text documents containing information search, machine translation, answering questions, and automatic summarization. This study aims to apply NER in Indonesian language news classification. This study uses Design-Based Research whose process includes (1) pre-implementation, (2) design, (3) implementation and revision, and finally, (4) reflection and evaluation. This application was developed on the platform python, streamlit, BeautifulSoup, gnews, and spacy library. The results of application accuracy testing have an F1-score value of 89.69% for all entities consisting of place, figure, day, date, and organization.

Text and Dynamic Network Analysis for Measuring Technological Convergence: A Case Study on Defense Patent Data

Identifying technology convergence is getting harder, especially in fast-evolving and cross-technological fields. On the other side, natural language processing and network analysis researchers is to provide a novel method for mapping technologies and their relations over time, in order to identify dynamic patterns of convergence and to test it on the C4ISTAR field, a defense-related cross-technological field. The methodology automatically extracts technologies from a corpus of scientific papers, policy documents, websites, and company reports using Named Entity Recognition approaches. These technologies are then retrieved from the second corpus of more than 300 thousand patents related to the C4ISTAR domain, measuring their cooccurrences over time. Finally, using the time-varying network analysis, we were able to identify and measure the pattern of technological convergence.

A novel neural network model fusion approach for improving medical named entity recognition in online health expert question-answering services

Because of the frequent occurrence of chronic diseases, the COVID-19 pandemic, etc., online health expert question-answering (HQA) services have been unable to cope with the rapidly increasing demand for online consultations. Building a virtual health assistant based on medical named entity recognition (NER) can effectively assist with the consultation process, but the unstandardized expressions within HQA text pose a serious challenge for medical NER tasks. The main goal of this study is to propose a novel deep medical NER approach based on a collaborative decision strategy (CDS), i.e., co_decision_NER (CDN), that can identify standard and nonstandard medical entities in the HQA context. We collected 10,000 question–answer pairs from HaoDF, extracted medical entities from 15 entity categories, and used a CDS to fuse the advantages of different NER models. Ultimately, CDN achieved a performance (precision = 84.50%, recall = 84.30%, F1 = 84.40%) that was significantly better than that of the state-of-the-art (SOTA) method. Our empirical analysis suggests that the entity types Disease (DIS), Sign (SIG), Test (TES), Drug (DRU), Surgery (SUR), Precaution (PRE), and Region (REG) can be most easily expressed arbitrarily in the doctor–patient interaction scenario of HQA services. In addition, CDN can identify not only standard but also nonstandard medical entities, effectively alleviating the severe out-of-vocabulary (OOV) problem faced by HQA services when performing medical NER tasks. The core contribution of this study is the development of a novel neural network model fusion algorithm that can improve the performance of entity recognition in medical domain-specific tasks.

A deep active learning-based and crowdsourcing-assisted solution for named entity recognition in Chinese historical corpora

PurposeThe majority of existing studies about named entity recognition (NER) concentrate on the prediction enhancement of deep neural network (DNN)-based models themselves, but the issues about the scarcity of training corpus and the difficulty of annotation quality control are not fully solved, especially for Chinese ancient corpora. Therefore, designing a new integrated solution for Chinese historical NER, including automatic entity extraction and man-machine cooperative annotation, is quite valuable for improving the effectiveness of Chinese historical NER and fostering the development of low-resource information extraction.Design/methodology/approachThe research provides a systematic approach for Chinese historical NER with a three-stage framework. In addition to the stage of basic preprocessing, the authors create, retrain and yield a high-performance NER model only using limited labeled resources during the stage of augmented deep active learning (ADAL), which entails three steps—DNN-based NER modeling, hybrid pool-based sampling (HPS) based on the active learning (AL), and NER-oriented data augmentation (DA). ADAL is thought to have the capacity to maintain the performance of DNN as high as possible under the few-shot constraint. Then, to realize machine-aided quality control in crowdsourcing settings, the authors design a stage of globally-optimized automatic label consolidation (GALC). The core of GALC is a newly-designed label consolidation model called simulated annealing-based automatic label aggregation (“SA-ALC”), which incorporates the factors of worker reliability and global label estimation. The model can assure the annotation quality of those data from a crowdsourcing annotation system.FindingsExtensive experiments on two types of Chinese classical historical datasets show that the authors’ solution can effectively reduce the corpus dependency of a DNN-based NER model and alleviate the problem of label quality. Moreover, the results also show the superior performance of the authors’ pipeline approaches (i.e. HPS + DA and SA-ALC) compared to equivalent baselines in each stage.Originality/valueThe study sheds new light on the automatic extraction of Chinese historical entities in an all-technological-process integration. The solution is helpful to effectively reducing the annotation cost and controlling the labeling quality for the NER task. It can be further applied to similar tasks of information extraction and other low-resource fields in theoretical and practical ways.

A Named Entity and Relationship Extraction Method from Trouble-Shooting Documents in Korean

In enterprises operating large-scale equipment, such as plants, maintenance workers must quickly and accurately find and understand the information in the equipment maintenance documents to perform maintenance tasks effectively. If the equipment maintenance documents include sentences with semantically ambiguous expressions, it will interfere with the maintenance knowledge search, and it may affect the maintenance performance of engineers. In order to solve these problems, text-based research of maintenance documents have been done to extract the key information or knowledge from these documents. Previous studies focused on finding the technical terminologies or calculating the similarity of documents using named entity recognition approaches. This paper proposes a method to extract knowledge of not only the technical terminologies but also their relations. The proposed method uses a rule-based approach that can be applied to the results of a named entity recognition approach and a dependency parsing approach. The named entity recognition approach found technical terms and the dependency parsing approach provided sentence structure information, so that the proposed method showed that a set of rules can extract maintenance knowledge, including entities and their relations. Trouble-shooting documents in the field were used as an experiment to demonstrate the effectiveness of the proposed method, and the experiment showed the possibility of practical use of the proposed method.