Named Entity Recognition in Chinese Electronic Medical Record Using Attention Mechanism

Abstract

In order to improve the accuracy of the NER (Named entity recognition) model on the Chinese EMR (Electronic Medical Record), we propose an effective deep learning model based on the attention mechanism and bootstrapping method to identify entity and relationship on the Chinese EMR. For the attention-based deep learning model, we first convert words into word vectors and input them into Bi-directional Long Short-Term Memory (BiLSTM). Then we adopt attention mechanism to assign different attention weights on hidden layer states at different words, thus the semantic features can focus on the words related to entity vocabulary, making up for the deficiencies of the BiLSTM model. Moreover, we employ the transformation matrix in Conditional Random Field (CRF) to solve the labeling bias problem in BiLSTM, thus we can obtain a global optimal tagging for the input sentences. Next, we use the bootstrapping methods to expand the labeled EMR corpus for overcoming the scarcity of the EMR corpus. Finally, the extensive experimental evaluation on Chinese EMR shows the superiority of the proposed approach under multiple evaluation criteria.