Named Entity Recognition Research Articles

Development and Validation of an AI-Driven System for Automatic Literature Analysis and Molecular Regulatory Network Construction.

Decoding gene regulatory networks is essential for understanding the mechanisms underlying many complex diseases. GENET is developed, an automated system designed to extract and visualize extensive molecular relationships from published biomedical literature. Using natural language processing, entities and relations are identified from a randomly selected set of 1788 scientific articles, and visualized in a filterable knowledge graph. The performance of GENET is evaluated and compared with existing methods. The named entity recognition model has achieved an overall precision of 94.23% (4835/5131; 93.56-94.84%), recall of 97.72% (4835/4948; 97.27-98.10%), and an F1 score of 95.94%. The relation extraction model has demonstrated an overall precision of 91.63% (2593/2830; 90.55-92.59%), recall of 89.17% (2593/2908; 87.99-90.25%), and an F1 score of 90.38%. GENET significantly outperforms existing methods in extracting molecular relationships (P<0.001). Additionally, GENET has successfully predicted WNT family member 4 regulates insulin-like growth factor 2 via signal transducer and activator of transcription 3 in colon cancer. With RNA sequencing data and multiple immunofluorescence, the authenticity of this prediction is validated, supporting the promising feasibility of GENET.

How Does a Generative Large Language Model Perform on Domain-Specific Information Extraction?─A Comparison between GPT-4 and a Rule-Based Method on Band Gap Extraction.

The advent of generative Large Language Models (LLMs) has greatly impacted the field of Natural Language Processing. However, it is inconclusive how generative LLMs perform on domain-specific information extraction tasks. This study compares the performance of GPT-4 and a rule-based information extraction method based on ChemDataExtractor on band gap information extraction, a task that has important implications for the materials science domain. No training data is required for either method, which is desirable because there is a lack of training data in the materials science domain compared with a variety of material information that is of interest. Manual evaluation on 415 randomly selected articles showed that the GPT-4 model achieved a higher level of accuracy in extracting materials' band gap information than the rule-based method (Correctness 87.95% vs 51.08%, Partial correctness 11.33% vs 36.87%, incorrectness 0.72% vs 12.05%). Further analysis of the errors reveals the strengths and weaknesses of the GPT-4 model compared to the rule-based method. The GPT-4 model shows stronger performance in interdependency resolution and complicated material name recognition, while it also has weaknesses in hallucination, identifying band gap values, and identifying band gap types. Revised prompt based on the error analysis leads to improved accuracy for GPT-4. To the best of our knowledge, this study is the first to compare the GPT-4 model and ChemDataExtractor for the band gap extraction task. This study provides evidence to support using generative LLMs for domain-specific information extraction tasks.

Text mining approach for feature extraction and cartilage disease grade classification using knee MRI radiology reports

MRI radiology reporting processes can be improved by exploiting structured and semantically labelled data that can be fed to artificial intelligence (AI) tools. AI-based tools assisting radiology reporting can help to automatically individuate cartilage grading in textual magnetic resonance imaging (MRI) reports, thus supporting clinicians' decisions regarding medical imaging utilisation, diagnosis and treatment. In this study, we extracted information (clinical findings, observations, anatomical regions, etc.) and classified knee cartilage degradation from medical reports utilising transfer-learning techniques applied to the Bidirectional Encoder Representations from Transformers (BERT) model and its variants, pre-trained on an Italian-language corpus. To realise this objective, we used a dataset of 750 MRI knee reports written by three radiologists who contributed to a manual annotation process to perform text classification (TC) and named entity recognition (NER) tasks. The dataset was obtained from an internal database of the IRCCS SYNLAB SDN. Seventy percent of the dataset was used for training, 10% was used for validation and 20% was used for testing. The best-performing configurations for NER and TC tasks were based on the pre-trained BERT model. The macro F1-scores obtained with the NER and TC models are 0.89 and 0.81, respectively. The accuracies calculated on the test set for both tasks are 0.96 and 0.99, respectively.

Business insights using RAG–LLMs: a review and case study

ABSTRACT As organizations increasingly rely on diverse data sources like invoices and surveys, efficient Information Extraction (IE) is crucial. Natural Language Processing (NLP) enhances IE through tasks such as Named Entity Recognition (NER), Relation Extraction (RE), Event Extraction (EE), Term Extraction (TE), and Topic Modeling (TM). However, implementing these methods requires significant expertise, which smaller organizations often lack. Large Language Models (LLMs), powered by Generative Artificial Intelligence (GenAI), can address this by performing multiple IE tasks without extensive development costs. However, LLMs may struggle with domain-specific accuracy. Integrating Retrieval-Augmented Generation (RAG) with LLMs improves precision by incorporating external data. Despite the potential, research on RAG-LLM applications in the business domain is limited. This article reviews Business IE systems, explores RAG-LLM applications across disciplines, and presents a case study demonstrating how RAG-LLMs can enhance business insights, offering scalable, cost-effective solutions.

Information Extraction of Aviation Accident Causation Knowledge Graph: An LLM-Based Approach

Summarizing the causation of aviation accidents is conducive to enhancing aviation safety. The knowledge graph of aviation accident causation, constructed based on aviation accident reports, can assist in analyzing the causes of aviation accidents. With the continuous development of artificial intelligence technology, leveraging large language models for information extraction and knowledge graph construction has demonstrated significant advantages. This paper proposes an information extraction method for aviation accident causation based on Claude-prompt, which relies on the large-scale pre-trained language model Claude 3.5. Through prompt engineering, combined with a few-shot learning strategy and a self-judgment mechanism, this method achieves automatic extraction of accident-cause entities and their relationships. Experimental results indicate that this approach effectively improves the accuracy of information extraction, overcoming the limitations of traditional methods in terms of accuracy and efficiency in processing complex texts. It provides strong support for subsequently constructing a structured knowledge graph of aviation accident causation and conducting causation analysis of aviation accidents.

Named Entity Recognition for Equipment Fault Diagnosis Based on RoBERTa-wwm-ext and Deep Learning Integration

Equipment fault diagnosis NER is to extract specific entities from Chinese equipment fault diagnosis text, which is the premise of constructing an equipment fault diagnosis knowledge graph. Named entity recognition for equipment fault diagnosis can also provide important data support for equipment maintenance support. Equipment fault diagnosis text has complex semantics, fuzzy entity boundaries, and limited data size. In order to extract entities from the equipment fault diagnosis text, this paper presents an NER model for equipment fault diagnosis based on RoBERTa-wwm-ext and Deep Learning network integration. Firstly, this model uses the RoBERTa-wwm-ext to extract context-sensitive embeddings of text sequences. Secondly, the context feature information is obtained through the BiLSTM network. Thirdly, the CRF is combined to output the label sequence with a constraint relationship, improve the accuracy of sequence labeling task, and complete the entity recognition task. Finally, experiments and predictions are carried out on the constructed dataset. The results show that the model can effectively identify five types of equipment fault diagnosis entities and has higher evaluation indexes than the traditional model. Its precision, recall, and F1 value are 94.57%, 95.39%, and 94.98%, respectively. The case study proves that the model can accurately recognize the entity of the input text.

Augmenting biomedical named entity recognition with general-domain resources

ObjectiveTraining a neural network-based biomedical named entity recognition (BioNER) model usually requires extensive and costly human annotations. While several studies have employed multi-task learning with multiple BioNER datasets to reduce human effort, this approach does not consistently yield performance improvements and may introduce label ambiguity in different biomedical corpora. We aim to tackle those challenges through transfer learning from easily accessible resources with fewer concept overlaps with biomedical datasets. MethodsWe proposed GERBERA, a simple-yet-effective method that utilized general-domain NER datasets for training. We performed multi-task learning to train a pre-trained biomedical language model with both the target BioNER dataset and the general-domain dataset. Subsequently, we fine-tuned the models specifically for the BioNER dataset. ResultsWe systematically evaluated GERBERA on five datasets of eight entity types, collectively consisting of 81,410 instances. Despite using fewer biomedical resources, our models demonstrated superior performance compared to baseline models trained with additional BioNER datasets. Specifically, our models consistently outperformed the baseline models in six out of eight entity types, achieving an average improvement of 0.9% over the best baseline performance across eight entities. Our method was especially effective in amplifying performance on BioNER datasets characterized by limited data, with a 4.7% improvement in F1 scores on the JNLPBA-RNA dataset. ConclusionThis study introduces a new training method that leverages cost-effective general-domain NER datasets to augment BioNER models. This approach significantly improves BioNER model performance, making it a valuable asset for scenarios with scarce or costly biomedical datasets. We make data, codes, and models publicly available via https://github.com/qingyu-qc/bioner_gerbera.

Integrating AI with medical industry chain data: enhancing clinical nutrition research through semantic knowledge graphs.

In clinical nutrition research, the medical industry chain generates a wealth of multidimensional spatial data across various formats, including text, images, and semi-structured tables. This data's inherent heterogeneity and diversity present significant challenges for processing and mining, which are further compounded by the data's diverse features, which are difficult to extract. To address these challenges, we propose an innovative integration of artificial intelligence (AI) with the medical industry chain data, focusing on constructing semantic knowledge graphs and extracting core features. These knowledge graphs are pivotal for efficiently acquiring insights from the vast and granular big data within the medical industry chain. Our study introduces the Clinical Feature Extraction Knowledge Mapping ( ) model, designed to augment the attributes of medical industry chain knowledge graphs through an entity extraction method grounded in syntactic dependency rules. The model is applied to real and large-scale datasets within the medical industry chain, demonstrating robust performance in relation extraction, data complementation, and feature extraction. It achieves superior results to several competitive baseline methods, highlighting its effectiveness in handling medical industry chain data complexities. By representing compact semantic knowledge in a structured knowledge graph, our model identifies knowledge gaps and enhances the decision-making process in clinical nutrition research.

Have we Reached a “Tipping Point” in Climate Change Reporting? How Mainstream Newspapers Cover Heatwaves

ABSTRACT This paper investigates the evolution of media coverage on heatwaves from 2010 to 2022 focusing on six countries (UK, USA, India, Australia, Canada, and China) through analysis of 27,964 newspaper reports. By employing content analysis and named entity recognition, we examine the volume of heatwave-related reports, their connection to climate change, and the incorporation of environmental science within media narratives. We explain our findings with reference to the concept of a dynamic or mobile public sphere. We identify three types of diffusion in the reporting of climate change in public spheres, which we term temporal diffusion, geographical diffusion, and political diffusion. These concepts illustrate the spread of ideas across time, space, and ideology. These shifts are characterised by an escalating media focus on heatwaves over time and the increasing association between heatwaves and anthropogenic climate change, a diffusion from countries in the Global North to the Global South, and a broadening attention from predominantly left-wing media across a wider ideological spectrum. This study contributes to our understanding of how journalistic practice, accumulated media attention, and environmental science can lead to significant shifts in transnationally linked public spheres, encouraging broader societal recognition of and engagement with climate change issues.

MediAlbertina: An European Portuguese medical language model

BackgroundPatient medical information often exists in unstructured text containing abbreviations and acronyms deemed essential to conserve time and space but posing challenges for automated interpretation. Leveraging the efficacy of Transformers in natural language processing, our objective was to use the knowledge acquired by a language model and continue its pre-training to develop an European Portuguese (PT-PT) healthcare-domain language model. MethodsAfter carrying out a filtering process, Albertina PT-PT 900M was selected as our base language model, and we continued its pre-training using more than 2.6 million electronic medical records from Portugal's largest public hospital. MediAlbertina 900M has been created through domain adaptation on this data using masked language modelling. ResultsThe comparison with our baseline was made through the usage of both perplexity, which decreased from about 20 to 1.6 values, and the fine-tuning and evaluation of information extraction models such as Named Entity Recognition and Assertion Status. MediAlbertina PT-PT outperformed Albertina PT-PT in both tasks by 4–6% on recall and f1-score. ConclusionsThis study contributes with the first publicly available medical language model trained with PT-PT data. It underscores the efficacy of domain adaptation and offers a contribution to the scientific community in overcoming obstacles of non-English languages. With MediAlbertina, further steps can be taken to assist physicians, in creating decision support systems or building medical timelines in order to perform profiling, by fine-tuning MediAlbertina for PT- PT medical tasks.

FE-CFNER: Feature Enhancement-based approach for Chinese Few-shot Named Entity Recognition

Although significant progress has been made in Chinese Named Entity Recognition (NER) methods based on deep learning, their performance often falls short in few-shot scenarios. Feature enhancement is considered a promising approach to address the issue of Chinese few-shot NER. However, traditional feature fusion methods tend to lead to the loss of important information and the integration of irrelevant information. Despite the benefits of incorporating BERT for improving entity recognition, its performance is limited when training data is insufficient. To tackle these challenges, this paper proposes a Feature Enhancement-based approach for Chinese Few-shot NER called FE-CFNER. FE-CFNER designs a double cross neural network to minimize information loss through the interaction of feature cross twice. Additionally, adaptive weights and a top-k mechanism are introduced to sparsify attention distributions, enabling the model to prioritize important information related to entities while excluding irrelevant information. To further enhance the quality of BERT embeddings, FE-CFNER employs a contrastive template for contrastive learning pre-training of BERT, enhancing BERT’s semantic understanding capability. We evaluate the proposed method on four sampled Chinese NER datasets: Weibo, Resume, Taobao, and Youku. Experimental results validate the effectiveness and superiority of FE-CFNER in Chinese few-shot NER tasks.

FRGEM: Feature integration pre-training based Gaussian embedding model for Chinese word representation

Most Chinese word representation approaches mine the lexeme of Chinese words by predicting the target word or the contextual words. However, Chinese words may have multiple meanings, above ways may only capture one aspect implication. In this paper, for better modeling multiple connotations of Chinese words, a Feature integRation pre-training based Gaussian Embedding Model (FRGEM) is proposed with three stages, feature integration pre-training, relevancy Gaussian representation and similarity-based training. In the first stage, the internal sequences with three inner-character stroke, structure and pinyin features of Chinese words are generated to learn their semantic relevancy, then we combine BERT with internal sequences to extract sentence relevance and integrate the internal and sentence features to learn pre-training representations. In the second stage, relevancy Gaussian representation with pre-training embeddings is proposed to fuse inner-character features with Gaussian through estimating its mean to analyze multiple implications of Chinese words. In the third stage, a similarity-based objective is proposed to distinguish true contextual words and learn Chinese word representations. Extensive experiments on word similarity, word analogy, named entity recognition and text classification show that FRGEM is superior to majority state-of-the-art algorithms.

From pre-training to fine-tuning: An in-depth analysis of Large Language Models in the biomedical domain

In this study, we delve into the adaptation and effectiveness of Transformer-based, pre-trained Large Language Models (LLMs) within the biomedical domain, a field that poses unique challenges due to its complexity and the specialized nature of its data. Building on the foundation laid by the transformative architecture of Transformers, we investigate the nuanced dynamics of LLMs through a multifaceted lens, focusing on two domain-specific tasks, i.e., Natural Language Inference (NLI) and Named Entity Recognition (NER). Our objective is to bridge the knowledge gap regarding how these models’ downstream performances correlate with their capacity to encapsulate task-relevant information. To achieve this goal, we probed and analyzed the inner encoding and attention mechanisms in LLMs, both encoder- and decoder-based, tailored for either general or biomedical-specific applications. This examination occurs before and after the models are fine-tuned across various data volumes. Our findings reveal that the models’ downstream effectiveness is intricately linked to specific patterns within their internal mechanisms, shedding light on the nuanced ways in which LLMs process and apply knowledge in the biomedical context. The source code for this paper is available at https://github.com/agnesebonfigli99/LLMs-in-the-Biomedical-Domain.

Construction and preliminary application of large language model for reservoir performance analysis

A large language model (LLM) is constructed to address the sophisticated demands of data retrieval and analysis, detailed well profiling, computation of key technical indicators, and the solutions to complex problems in reservoir performance analysis (RPA). The LLM is constructed for RPA scenarios with incremental pre-training, fine-tuning, and functional subsystems coupling. Functional subsystem and efficient coupling methods are proposed based on named entity recognition (NER), tool invocation, and Text-to-SQL construction, all aimed at resolving pivotal challenges in developing the specific application of LLMs for RDA. This study conducted a detailed accuracy test on feature extraction models, tool classification models, data retrieval models and analysis recommendation models. The results indicate that these models have demonstrated good performance in various key aspects of reservoir dynamic analysis. The research takes some injection and production well groups in the PK3 Block of the Daqing Oilfield as an example for testing. Testing results show that our model has significant potential and practical value in assisting reservoir engineers with RDA. The research results provide a powerful support to the application of LLM in reservoir performance analysis.

TourismNER: A Tourism Named Entity Recognition method based on entity boundary joint prediction

Tourism named entity recognition is indispensable in tourism information extraction, and plays a crucial role in constructing tourism knowledge map and enhancing tourism knowledge quiz system. The difficulty of tourism named entity recognition lies in its complex nested structure, and the lengthy entity naming length. To address these existing problems, we propose a tourism named entity recognition model that jointly predicts entity boundaries, adopting a training strategy of data preprocessing to enhance the model’s ability for tourism named entity boundary recognition, while our model introduces a pre-trained Bert model as well as BiLSTM coding to enhance the representation of the model’s contexts, and uses a combined predictor of Biaffine and MLP to enhance the model’s recognition performance for boundaries, as well as introducing label smoothing cross entropy to smooth the target labels during the training process. Experiments are conducted on three datasets with different granularities. From the analysis of the experimental results, it can be seen that the named entity recognition method achieves higher accuracy and F1 value compared with the optimal baseline model, and also proves the effectiveness and generality of the modeling method proposed in this paper.

Chinese Named Entity Recognition based on adaptive lexical weights

Currently, many researchers use weights to merge self-matched words obtained through dictionary matching in order to enhance the performance of Named Entity Recognition (NER). However, these studies overlook the relationship between words and sentences when calculating lexical weights, resulting in fused word information that often does not align with the intended meaning of the sentence. Addressing above issue and enhance the prediction performance, we propose an adaptive lexical weight approach for determining lexical weights. Given a sentence, we utilize an enhanced global attention mechanism to compute the correlation between self-matching words and sentences, thereby focusing attention on crucial words while disregarding unreliable portions. Experimental results demonstrate that our proposed model outperforms existing state-of-the-art methods for Chinese NER of MRSA, Weibo, and Resume datasets.

Uncovering condition information loss in medical text extraction: The challenge of non-contiguous spans

We investigated the limitations of conventional named entity recognition (NER) and entity linking (EL) methods in accurately extracting patient condition information from medical texts, focusing on the challenges posed by non-contiguous spans and the potential information loss. We utilized a corpus with entity-relation annotations, analyzing the frequency and nature of non-contiguous spans that include irrelevant entities within gaps. The corpus was further analyzed to pinpoint the types of entity representations predominantly linked with peripheral spans—those not encompassing central symptom-describing terms—with a focus on items, body parts, and clinical tests. Our analysis revealed that 18.6 % of patient condition expressions were non-contiguous spans containing irrelevant entities, suggesting an accuracy ceiling of 81.4 % for conventional NER and EL approaches in the worst-case scenario. The study highlights the importance of entity types such as items, body parts, and clinical tests in these expressions, indicating that conventional extraction methods incur considerable information loss. The findings underscore the need for more sophisticated information extraction techniques capable of handling the complexities of medical texts, including non-contiguous spans. Adapting methods that allow gaps within entities or employing graph-based term assignments can enhance the accuracy and comprehensiveness of medical text annotation.

P2.05B.02 Automated Extraction of Key Entities from Thorax CT Reports Using NER with Prompt Engineering

P2.05B.02 Automated Extraction of Key Entities from Thorax CT Reports Using NER with Prompt Engineering

BAMRE: Joint extraction model of Chinese medical entities and relations based on Biaffine transformation with relation attention

Electronic Health Records (EHRs) contain various valuable medical entities and their relationships. Although the extraction of biomedical relationships has achieved good results in the mining of electronic health records and the construction of biomedical knowledge bases, there are still some problems. There may be implied complex associations between entities and relationships in overlapping triplets, and ignoring these interactions may lead to a decrease in the accuracy of entity extraction. To address this issue, a joint extraction model for medical entity relations based on a relation attention mechanism is proposed. The relation extraction module identifies candidate relationships within a sentence. The attention mechanism based on these relationships assigns weights to contextual words in the sentence that are associated with different relationships. Additionally, it extracts the subject and object entities. Under a specific relationship, entity vector representations are utilized to construct a global entity matching matrix based on Biaffine transformations. This matrix is designed to enhance the semantic dependencies and relational representations between entities, enabling triplet extraction. This allows the two subtasks of named entity recognition and relation extraction to be interrelated, fully utilizing contextual information within the sentence, and effectively addresses the issue of overlapping triplets.Experimental observations from the CMeIE Chinese medical relation extraction dataset and the Baidu2019 Chinese dataset confirm that our approach yields the superior F1 score across all cutting-edge baselines. Moreover, it offers substantial performance improvements in intricate situations involving diverse overlapping patterns, multitudes of triplets, and cross-sentence triplets.

Character-Level Embedding using FastText and LSTM for Biomedical Named Entity Recognition

Extracting biomedical entities has caught many researchers’ attention in which the recent technique of word embedding is employed for such a task. Yet, the traditional word embedding architectures of Word2vec or Glove are still suffering from the ‘out-of-vocabulary’ (OOV) problem. This problem occurs when an unseen term might be encountered during the testing which leads to absence of embedding vector. Hence, this study aims to propose a character-level embedding through FastText architecture. In fact, handling the character-level seems a promising solution for the OOV problem. To this end, the proposed FastText architecture has been used to generate embedding vectors for the possible N-gram combinations of each word. Consequentially, these vectors have been fed to a Long Short Term Memory (LSTM) architecture for classifying the words into its biomedical classes. Using two benchmark datasets of BioCreative-II and NCBI, the proposed method was able to produce an f-measure of 0.912 and 0.918 respectively. Comparing these results with the baseline studies demonstrates the superiority of the proposed character-level embedding of FastText in terms of Biomedical Named Entity Recognition (BNER) task.