Evaluation of Transfer Learning for Adverse Drug Event (ADE) and Medication Entity Extraction

Sankaran Narayanan,Kaivalya Mannam,P Venkat Rangan,Sreeranga P Rajan

doi:10.18653/v1/2020.clinicalnlp-1.6

Abstract

We evaluate several biomedical contextual embeddings (based on BERT, ELMo, and Flair) for the detection of medication entities such as Drugs and Adverse Drug Events (ADE) from Electronic Health Records (EHR) using the 2018 ADE and Medication Extraction (Track 2) n2c2 data-set. We identify best practices for transfer learning, such as language-model fine-tuning and scalar mix. Our transfer learning models achieve strong performance in the overall task (F1=92.91%) as well as in ADE identification (F1=53.08%). Flair-based embeddings out-perform in the identification of context-dependent entities such as ADE. BERT-based embeddings out-perform in recognizing clinical terminology such as Drug and Form entities. ELMo-based embeddings deliver competitive performance in all entities. We develop a sentence-augmentation method for enhanced ADE identification benefiting BERT-based and ELMo-based models by up to 3.13% in F1 gains. Finally, we show that a simple ensemble of these models out-paces most current methods in ADE extraction (F1=55.77%).

Highlights

Adverse Drug Events (ADE) arising from the medical intervention of drugs account for 1.3 million visits to the emergency department in the United States alone (CDC, 2017)
Following (Miller et al, 2019), we modeled this as a named-entity recognition task
We evaluate transfer learning models incorporating: BioBERT (Lee et al, 2020), ClinicalBERT (Alsentzer et al, 2019), ELMo (Peters et al, 2018) and Flair (Akbik et al, 2018) contextual embeddings pre-trained on PubMed abstracts (Fiorini et al, 2018)

Summary

Introduction

Randomized controlled trials (RCTs), the primary mechanism for monitoring and identifying ADEs, are hampered by insufficient sample sizes of clinical trials (Sultana et al, 2013) Pharmacovigilance databases such as the Food and Drug Administration’s Adverse Event Reporting System (FAERS) strive to be authoritative sources for Physicians; they require regular manual data entry (Hoffman et al, 2014; Chedid et al, 2018). In the example given, ’seizure prophylaxis’ and ’few days’ may occur any where in the clinical text, but only in the context of ’Dilantin’ they indicate reason / duration for administration Such ‘dynamic’ interfaces can aid medical students to learn from their collective experiences

Methods

Findings

Discussion

Conclusion