Med-BERT: A Pretraining Framework for Medical Records Named Entity Recognition

Abstract

A large amount of data is generated every day with the development of Internet medical care, which is of great significance for the clinical decision support system and medical real-world research. Medical records named entity recognition (NER) is important on the aforementioned research topics under the premise of protecting patients’ private information. In this article, we propose a medical dictionary enhanced bidirectional encoder representations from transformers (BERT), dubbed Med-BERT, to achieve better representations of long medical entities. On Med-BERT, we propose a span flat-lattice transformer (Span-FLAT) method on medical records NER, and the entity types include private information such as names and addresses, as well as medical information such as patient symptoms, signs, and diseases. Experimental results on two benchmark medical datasets show the effectiveness of Med-BERT, and the proposed Med-BERT-based Span-FLAT method remarkably outperforms the state-of-the-art methods on medical NER task.