Fine-Tuning Transformer Models for Adverse Drug Event Identification and Extraction in Biomedical Corpora: A Comparative Study

Abstract

Adverse Drug Events (ADEs) are potentially fatal problems that patients can deal with only if they have a solid awareness of them. With the available amount of unstructured textual data from biomedical literature, electronic records, and social media (e.g., tweets), early detection of unfavorable reactions and sharing them with biomedical experts, pharma companies, and healthcare professionals is a necessity, as this can prevent morbidity and save many lives. The Biomedical Named Entity Recognition (BioNER) task can be considered the initial step toward resolving this issue. In this paper, we present an empirical evaluation experiment by fine-tuning pretrained language models for detecting biomedical entities (e.g., drug-names and symptoms). We fine-tuned five transformer models: BERT (Bidirectional Encoder Representations from Transformers), SpanBERT, BioBERT, BlueBERT, and SCIBERT, on two well-known biomedical datasets, CADEC and ADE-corpus. The evaluation results demonstrate that BioBERT which was pretrained on both general and domain-specific (biomedical domain) corpora outperformed all other models on both datasets and reached 90.3% and 68,73% on the F1-score in the ADE and CADEC corpora, respectively.