Biomedical Named Entity Recognition Research Articles

Advancing entity recognition in biomedicine via instruction tuning of large language models.

Large Language Models (LLMs) have the potential to revolutionize the field of Natural Language Processing, excelling not only in text generation and reasoning tasks but also in their ability for zero/few-shot learning, swiftly adapting to new tasks with minimal fine-tuning. LLMs have also demonstrated great promise in biomedical and healthcare applications. However, when it comes to Named Entity Recognition (NER), particularly within the biomedical domain, LLMs fall short of the effectiveness exhibited by fine-tuned domain-specific models. One key reason is that NER is typically conceptualized as a sequence labeling task, whereas LLMs are optimized for text generation and reasoning tasks. We developed an instruction-based learning paradigm that transforms biomedical NER from a sequence labeling task into a generation task. This paradigm is end-to-end and streamlines the training and evaluation process by automatically repurposing pre-existing biomedical NER datasets. We further developed BioNER-LLaMA using the proposed paradigm with LLaMA-7B as the foundational LLM. We conducted extensive testing on BioNER-LLaMA across three widely recognized biomedical NER datasets, consisting of entities related to diseases, chemicals, and genes. The results revealed that BioNER-LLaMA consistently achieved higher F1-scores ranging from 5% to 30% compared to the few-shot learning capabilities of GPT-4 on datasets with different biomedical entities. We show that a general-domain LLM can match the performance of rigorously fine-tuned PubMedBERT models and PMC-LLaMA, biomedical-specific language model. Our findings underscore the potential of our proposed paradigm in developing general-domain LLMs that can rival SOTA performances in multi-task, multi-domain scenarios in biomedical and health applications. Datasets and other resources are available at https://github.com/BIDS-Xu-Lab/BioNER-LLaMA.

Partial Annotation Learning for Biomedical Entity Recognition.

Named Entity Recognition (NER) is a key task to support biomedical research. In Biomedical Named Entity Recognition (BioNER), obtaining high-quality expert annotated data is laborious and expensive, leading to the development of automatic approaches such as distant supervision. However, manually and automatically generated data often suffer from the unlabeled entity problem, whereby many entity annotations are missing, degrading the performance of full annotation NER models. To conquer this issue, we undertake a systematic exploration of the efficacy of partial annotation learning methods for BioNER, which encompasses a comprehensive evaluation conducted across a spectrum of distinct simulated scenarios of missing entity annotations. Furthermore, we propose a TS-PubMedBERT-Partial-CRF partial annotation learning model. We standardize a compilation of 16 BioNER corpora, encompassing a range of five distinct entity types, to establish a gold standard. And we compare against the state-of-the-art partial annotation model EER-PubMedBERT, the widely acknowledged partial annotation model BiLSTM-Partial-CRF model, and the state-of-the-art full annotation learning BioNER model PubMedBERT tagger. Results show that partial annotation learning-based methods can effectively learn from biomedical corpora with missing entity annotations. Our proposed model outperforms alternatives and, specifically, the PubMedBERT tagger by 38% in F1-score under high missing entity rates. Moreover, the recall of entity mentions in our model demonstrates a competitive alignment with the upper threshold observed on the fully annotated dataset. We have published our data, source code and training records at https://github.com/possible1402/partial\_annotation\_learning.

RoBGP: A Chinese Nested Biomedical Named Entity Recognition Model Based on RoBERTa and Global Pointer

RoBGP: A Chinese Nested Biomedical Named Entity Recognition Model Based on RoBERTa and Global Pointer

Dictionary-based matching graph network for biomedical named entity recognition

Biomedical named entity recognition (BioNER) is an essential task in biomedical information analysis. Recently, deep neural approaches have become widely utilized for BioNER. Biomedical dictionaries, implemented through a masked manner, are frequently employed in these methods to enhance entity recognition. However, their performance remains limited. In this work, we propose a dictionary-based matching graph network for BioNER. This approach utilizes the matching graph method to project all possible dictionary-based entity combinations in the text onto a directional graph. The network is implemented coherently with a bi-directional graph convolutional network (BiGCN) that incorporates the matching graph information. Our proposed approach fully leverages the dictionary-based matching graph instead of a simple masked manner. We have conducted numerous experiments on five typical Bio-NER datasets. The proposed model shows significant improvements in F1 score compared to the state-of-the-art (SOTA) models: 2.8% on BC2GM, 1.3% on BC4CHEMD, 1.1% on BC5CDR, 1.6% on NCBI-disease, and 0.5% on JNLPBA. The results show that our model, which is superior to other models, can effectively recognize natural biomedical named entities.

A BIGRU-Based Stacked Attention Network for Biomedical Named Entity Recognition with Chinese EMRs.

Biomedical named entity recognition (BNER) is an effective method to structure the medical text data. It is an important basic task for building the medical application services such as the medical knowledge graphs and the intelligent auxiliary diagnosis systems. Existing medical named entity recognition methods generally leverage the word embedding model to construct text representation, and then integrate multiple semantic understanding models to enhance the semantic understanding ability of the model to achieve high-performance entity recognition. However, in the medical field, there are many professional terms that rarely appear in the general field, which cannot be represented well by the general domain word embedding model. Second, existing approaches typically only focus on the extraction of global semantic features, which generate a loss of local semantic features between characters. Moreover, as the word embedding dimension becomes much higher, the standard single-layer structure fails to fully and deeply extract the global semantic features. We put forward the BIGRU-based Stacked Attention Network (BSAN) model for biomedical named entity recognition. Firstly, we use the large-scale real-world medical electronic medical record (EMR) data to fine-tune BERT to build the proprietary embedding representations of the medical terms. Second, we use the Convolutional Neural Network model to extract semantic features. Finally, a stacked BIGRU is constructed using a multi-layer structure and a novel stacking method. It not only enables comprehensive and in-depth extraction of global semantic features, but also requires less time. Experimentally validated on the real-world datasets in Chinese EMRs, the proposed BSAN model achieves 90.9% performance on F1-values, which is stronger than the BNER performance of other state-of-the-art models.

A dictionary-guided attention network for biomedical named entity recognition in Chinese electronic medical records

A dictionary-guided attention network for biomedical named entity recognition in Chinese electronic medical records

MMBERT: a unified framework for biomedical named entity recognition.

Named entity recognition (NER) is an important task in natural language processing (NLP). In recent years, NER has attracted much attention in the biomedical field. However, due to the lack of biomedical named entity identification datasets, the complexity and rarity of biomedical named entities and so on, biomedical NER is more difficult than general domain NER. So in this paper, we propose a framework (MMBERT) based on Transformer to solve the problems above. To address the scarcity of biomedical named entity recognition datasets, we introduce ERNIE-Health, a new Chinese language representation model pre-trained on large-scale biomedical text corpora. Because of the complexity and rarity of biomedical named entities, we use the Bert and CW-LSTM structures to get the joint feature vector of word pairs relations. In addition, we design multi-granularity 2D convolution to refine the relationship and representation between word pairs. Finally, we design a convolutional neural network (CNN) structure and a co-predictor to improve the model's generalization capability and prediction accuracy. We have conducted extensive experiments on three benchmark datasets, and the experimental results show that our model achieves the best results compared with several baseline models in the experiment.

Data augmentation via context similarity: An application to biomedical Named Entity Recognition

In this paper, we present COntext SImilarity-based data augmentation for NER (COSINER), a new method for improving Named Entity Recognition (NER) tasks using data augmentation. Unlike current techniques, which may generate noisy and mislabeled samples through text manipulation, COSINER uses context similarity to replace entity mentions with more plausible ones on the basis of available training data and considering the context in which entities typically appear. Through experiments on popular benchmark datasets, we show that COSINER outperforms existing baselines in various few-shot scenarios where training data is limited. Additionally, our method’s computing times are comparable to the simplest augmentation methods and are better than approaches that rely on pre-trained models in their architecture.

Biomedical named entity recognition using TCN approaches and bio tagging

<p>Biomedical named entity recognition (BNER) is to identify instances in biomedical field such as chemical compounds, drugs, genes, RNA, DNA and proteins used in extracting information. It extracts relation between various drugs and their usage, profiles of similar and related drugs with help of machine learning approach. The efficiency in biomedical field is still in research for further improvement even many supervised methods are applied. The proposed method combines two algorithms and improve performance based on features used. It uses conditional random field (CRF) for entity identification and classification of temporal conventional network (TCN) to detect and recognize subtypes in BNER. Datasets such as GENIA and CHEMDNER corpus are used for evaluation with different entity types. Results shows that proposed methods performed better compared to other machine learning approach. The detailed study of TCN has been discussed. The classification of BNER is mapped with various classification methods to enhance result of high recognition.</p>

An Effective Biomedical Named Entity Recognition by Handling Imbalanced Data Sets Using Deep Learning and Rule-Based Methods

An Effective Biomedical Named Entity Recognition by Handling Imbalanced Data Sets Using Deep Learning and Rule-Based Methods

Few-shot biomedical named entity recognition via knowledge-guided instance generation and prompt contrastive learning.

Few-shot learning (FSL) that can effectively perform named entity recognition in low-resource scenarios has raised growing attention, but it has not been widely studied yet in the biomedical field. In contrast to high-resource domains, biomedical named entity recognition (BioNER) often encounters limited human-labeled data in real-world scenarios, leading to poor generalization performance when training only a few labeled instances. Recent approaches either leverage cross-domain high-resource data or fine-tune the pre-trained masked language model using limited labeled samples to generate new synthetic data, which is easily stuck in domain shift problems or yields low-quality synthetic data. Therefore, in this paper, we study a more realistic scenario, i.e., few-shot learning for BioNER. Leveraging the domain knowledge graph, we propose knowledge-guided instance generation for few-shot BioNER, which generates diverse and novel entities based on similar semantic relations of neighbor nodes. In addition, by introducing question prompt, we cast BioNER as question answering (QA) task and propose prompt contrastive learning to improve the robustness of the model by measuring the mutual information (MI) between query-answer pairs. Extensive experiments conducted on various few-shot settings show that the proposed framework achieves superior performance. Particularly, in a low-resource scenario with only 20 samples, our approach substantially outperforms recent state-of-the-art (SoTA) models on four benchmark datasets, achieving an average improvement of up to 7.1% F1. Our source code and data are available at https://github.com/cpmss521/KGPC. Supplementary data are available at Bioinformatics online.

Biomedical-named entity recognition using CUDA accelerated KNN algorithm

Biomedical named entity recognition (Bio-NER) is a highly complex and time-consuming research domain using natural language processing (NLP). It’s widely used in information retrieval, knowledge summarization, biomolecular event extraction, and discovery applications. This paper proposes a method for the recognition and classification of named entities in the biomedical domain using machine learning (ML) techniques. Support vector machine (SVM), decision trees (DT), K-nearest neighbor (KNN), and its kernel versions are used. However, recent advancements in programmable, massively parallel graphics processing units (GPU) hold promise in terms of increased computational capacity at a lower cost to address multi-dimensional data and time complexity. We implement a novel parallel version of KNN by porting the distance computation step on GPU using the compute unified device architecture (CUDA) and compare the performance of all the algorithms using the BioNLP/NLPBA 2004 corpus. Results demonstrate that CUDA-KNN takes full advantage of the GPU’s computational capacity and multi-leveled memory architecture, resulting in a 35× performance enhancement over the central processing unit (CPU). In a comparative study with existing research, the proposed model provides an option for a faster NER system for higher dimensionality and larger datasets as it offers balanced performance in terms of accuracy and speed-up, thus providing critical design insights into developing a robust BioNLP system.

Transferring from Textual Entailment to Biomedical Named Entity Recognition.

Biomedical Named Entity Recognition (BioNER) aims at identifying biomedical entities such as genes, proteins, diseases, and chemical compounds in the given textual data. However, due to the issues of ethics, privacy, and high specialization of biomedical data, BioNER suffers from the more severe problem of lacking in quality labeled data than the general domain especially for the token-level. Facing the extremely limited labeled biomedical data, this work studies the problem of gazetteer-based BioNER, which aims at building a BioNER system from scratch. It needs to identify the entities in the given sentences when we have zero token-level annotations for training. Previous works usually use sequential labeling models to solve the NER or BioNER task and obtain weakly labeled data from gazetteers when we don't have full annotations. However, these labeled data are quite noisy since we need the labels for each token and the entity coverage of the gazetteers is limited. Here we propose to formulate the BioNER task as a Textual Entailment problem and solve the task via Textual Entailment with Dynamic Contrastive learning (TEDC). TEDC not only alleviates the noisy labeling issue, but also transfers the knowledge from pre-trained textual entailment models. Additionally, the dynamic contrastive learning framework contrasts the entities and non-entities in the same sentence and improves the model's discrimination ability. Experiments on two real-world biomedical datasets show that TEDC can achieve state-of-the-art performance for gazetteer-based BioNER.

Provision and Characterization of a Corpus for Pharmaceutical, Biomedical Named Entity Recognition for Pharmacovigilance: Evaluation of Language Registers and Training Data Sufficiency.

Machine learning (ML) systems are widely used for automatic entity recognition in pharmacovigilance. Publicly available datasets do not allow the use of annotated entities independently, focusing on small entity subsets or on single language registers (informal or scientific language). The objective of the current study was to create a dataset that enables independent usage of entities, explores the performance of predictive ML models on different registers, and introduces a method to investigate entity cut-off performance. A dataset has been created combining different registers with 18 different entities. We applied this dataset to compare the performance of integrated models with models created with single language registers only. We introduced fractional stratified k-fold cross-validation to determine model performance on entity level by using training dataset fractions. We investigated the course of entity performance with fractions of training datasets and evaluated entity peak and cut-off performance. The dataset combines 1400 records (scientific language: 790; informal language: 610) with 2622 sentences and 9989 entity occurrences and combines data from external (801 records) and internal sources (599 records). We demonstrated that single language register models underperform compared to integrated models trained with multiple language registers. A manually annotated dataset with a variety of different pharmaceutical and biomedical entities was created and is made available to the research community. Our results show that models that combine different registers provide better maintainability, have higher robustness, and have similar or higher performance. Fractional stratified k-fold cross-validation allows the evaluation of training data sufficiency on the entity level.

Acute stroke CDS: automatic retrieval of thrombolysis contraindications from unstructured clinical letters.

Thrombolysis treatment for acute ischaemic stroke can lead to better outcomes if administered early enough. However, contraindications exist which put the patient at greater risk of a bleed (e.g. recent major surgery, anticoagulant medication). Therefore, clinicians must check a patient's past medical history before proceeding with treatment. In this work we present a machine learning approach for accurate automatic detection of this information in unstructured text documents such as discharge letters or referral letters, to support the clinician in making a decision about whether to administer thrombolysis. We consulted local and national guidelines for thrombolysis eligibility, identifying 86 entities which are relevant to the thrombolysis decision. A total of 8,067 documents from 2,912 patients were manually annotated with these entities by medical students and clinicians. Using this data, we trained and validated several transformer-based named entity recognition (NER) models, focusing on transformer models which have been pre-trained on a biomedical corpus as these have shown most promise in the biomedical NER literature. Our best model was a PubMedBERT-based approach, which obtained a lenient micro/macro F1 score of 0.829/0.723. Ensembling 5 variants of this model gave a significant boost to precision, obtaining micro/macro F1 of 0.846/0.734 which approaches the human annotator performance of 0.847/0.839. We further propose numeric definitions for the concepts of name regularity (similarity of all spans which refer to an entity) and context regularity (similarity of all context surrounding mentions of an entity), using these to analyse the types of errors made by the system and finding that the name regularity of an entity is a stronger predictor of model performance than raw training set frequency. Overall, this work shows the potential of machine learning to provide clinical decision support (CDS) for the time-critical decision of thrombolysis administration in ischaemic stroke by quickly surfacing relevant information, leading to prompt treatment and hence to better patient outcomes.

Consistency enhancement of model prediction on document-level named entity recognition.

Biomedical named entity recognition (NER) plays a crucial role in extracting information from documents in biomedical applications. However, many of these applications require NER models to operate at a document level, rather than just a sentence level. This presents a challenge, as the extension from a sentence model to a document model is not always straightforward. Despite the existence of document NER models that are able to make consistent predictions, they still fall short of meeting the expectations of researchers and practitioners in the field. To address this issue, we have undertaken an investigation into the underlying causes of inconsistent predictions. Our research has led us to believe that the use of adjectives and prepositions within entities may be contributing to low label consistency. In this article, we present our method, ConNER, to enhance a label consistency of modifiers such as adjectives and prepositions. By refining the labels of these modifiers, ConNER is able to improve representations of biomedical entities. The effectiveness of our method is demonstrated on four popular biomedical NER datasets. On three datasets, we achieve a higher F1 score than the previous state-of-the-art model. Our method shows its efficacy on two datasets, resulting in 7.5%-8.6% absolute improvements in the F1 score. Our findings suggest that our ConNER method is effective on datasets with intrinsically low label consistency. Through qualitative analysis, we demonstrate how our approach helps the NER model generate more consistent predictions. Our code and resources are available at https://github.com/dmis-lab/ConNER/.

Development and Study of a Knowledge Graph for Retrieving the Relationship Between BVDV and Related Genes

Background: Bovine viral diarrhea virus (BVDV) can cause diarrhea, abortion, and immunosuppression in cattle, imposing huge economic losses for the global cattle industry. The pathogenic and immune mechanisms of BVDV remain elusive. The development of a BVDV-gene knowledge base can provide clues to reveal the interaction of BVDV with host cells. However, the traditional method of manually establishing a knowledge base is time-consuming and inefficient. The method of developing a knowledge base based on deep learning has noticeably attracted scholars' attention recently. Objective: The study aimed to explore the substitution of deep learning for manual mining of BVDVrelated genes and to develop a knowledge graph of the relationship between BVDV and related genes. Methods: A deep learning-based biomedical knowledge graph development method was proposed, which used deep learning to mine biomedical knowledge, model BVDV and various gene concepts, and store data in a graphical database. First, the PubMed database was used as the data source and crawler technology to obtain abstract data on the relationship between BVDV and various host genes. Pretrained BioBERT model was used for biomedical named entity recognition to obtain all types of gene entities, and the pre-trained BERT model was utilized for relationship extraction to achieve the relationship between BVDV and various gene entities. Then, it was combined with manual proofreading to obtain structured triple data with high accuracy. Finally, the Neo4j graph database was used to store data and to develop the knowledge graph of the relationship between BVDV and related genes. Results: The results showed the obtainment of 71 gene entity types, including PRL4, MMP-7, TGIF1, etc. 9 relation types of BVDV and gene entities were obtained, including "can downregulate expression of", "can upregulate expression of", "can suppress expression of", etc. The knowledge graph was developed using deep learning to mine biomedical knowledge combined with manual proofreading, which was faster and more efficient than the traditional method of establishing knowledge base manually, and the retrieval of semantic information by storing data in graph database was also more efficient. Conclusion: A BVDV-gene knowledge graph was preliminarily developed, which provided a basis for studying the interaction between BVDV and host cells.

Reinforcement learning based distantly supervised biomedical named entity recognition

Data used by current Biomedical named entity recognition (BioNER) systems has mostly been manually labelled for supervision. However, it might be difficult to find large amounts of annotated data, especially in fields with a high level of specialization, such as biomedical, bioinformatics, and so on. When dictionaries and ontologies are available, which are domain-specific knowledge resources, automatically tagged distantly supervised biomedical training data can be developed. However, any such distantly supervised NER result is normally noisy. The prevalence of false positives and false negatives with this type of autonomously generated data is the main problem that directly affects efficiency. This research investigates distant supervision to detect false positive occurrences in BioNER task. A reinforcement learning technique is employed that is modelled as a graphical processing unit (GPU) accelerated Markov decision process (MDP) with a neural network policy. To deal with false negative cases, we employ a partial annotation conditional random field (CRF) technique. Results on two benchmark datasets show a cutting-edge methodology that can enhance the functionality of the neural NER system. It goes on to show how the proposed approach cuts down on human annotated data for BioNER tasks in Natural Language Processing (NLP).

AIONER: all-in-one scheme-based biomedical named entity recognition using deep learning.

Biomedical named entity recognition (BioNER) seeks to automatically recognize biomedical entities in natural language text, serving as a necessary foundation for downstream text mining tasks and applications such as information extraction and question answering. Manually labeling training data for the BioNER task is costly, however, due to the significant domain expertise required for accurate annotation. The resulting data scarcity causes current BioNER approaches to be prone to overfitting, to suffer from limited generalizability, and to address a single entity type at a time (e.g. gene or disease). We therefore propose a novel all-in-one (AIO) scheme that uses external data from existing annotated resources to enhance the accuracy and stability of BioNER models. We further present AIONER, a general-purpose BioNER tool based on cutting-edge deep learning and our AIO schema. We evaluate AIONER on 14 BioNER benchmark tasks and show that AIONER is effective, robust, and compares favorably to other state-of-the-art approaches such as multi-task learning. We further demonstrate the practical utility of AIONER in three independent tasks to recognize entity types not previously seen in training data, as well as the advantages of AIONER over existing methods for processing biomedical text at a large scale (e.g. the entire PubMed data). The source code, trained models and data for AIONER are freely available at https://github.com/ncbi/AIONER.

Knowledge Adaptive Multi-Way Matching Network for Biomedical Named Entity Recognition via Machine Reading Comprehension.

Rapid and effective utilization of biomedical literature is paramount to combat diseases like COVID19. Biomedical named entity recognition (BioNER) is a fundamental task in text mining that can help physicians accelerate knowledge discovery to curb the spread of the COVID-19 epidemic. Recent approaches have shown that casting entity extraction as the machine reading comprehension task can significantly improve model performance. However, two major drawbacks impede higher success in identifying entities (1) ignoring the use of domain knowledge to capture the context beyond sentences and (2) lacking the ability to deeper understand the intent of questions. In this paper, to remedy this, we introduce and explore external domain knowledge which cannot be implicitly learned in text sequence. Previous works have focused more on text sequence and explored little of the domain knowledge. To better incorporate domain knowledge, a multi-way matching reader mechanism is devised to model representations of interaction between sequence, question and knowledge retrieved from Unified Medical Language System (UMLS). Benefiting from these, our model can better understand the intent of questions in complex contexts. Experimental results indicate that incorporating domain knowledge can help to obtain competitive results across 10 BioNER datasets, achieving absolute improvement of up to 2.02% in the f1 score.