Incorporating dynamic attention gating mechanism and pre-trained embedding for Chinese clinical named entity recognition

Abstract

Clinical Named Entity Recognition (CNER) serves as a cornerstone in clinical research and healthcare informatics, with its primary goal being the discernment and categorization of clinical terminologies present within Electronic Medical Records (EMRs). While deep learning has significantly improved CNER's performance, these models still face challenges in identifying rare or unseen entities. Our study unveils an innovative DAGM-BiLSTM-CRF model that seamlessly integrates data-driven deep learning strategies with human knowledge to surmount this limitation. The distinctive features of this model lie in its integration of a pre-trained character embedding layer, adept at encapsulating nuanced semantic variations at the granularity of individual characters. Furthermore, it incorporates a bidirectional Long Short-Term Memory (BiLSTM) layer, proficient in deriving contextually relevant information by traversing the sequence from both endpoints. The most noteworthy aspect of our model is the inclusion of a Dynamic Attention Gating Mechanism (DAGM) that dynamically recalibrates attention weights based on the traits of the input sequence and a Conditional Random Fields (CRF) layer that considers interdependencies among output labels within a sequence. The evaluation was carried out on the CCKS-2017 CNER benchmark dataset to underscore the superiority of our model over existing cutting-edge methods, thereby validating its proficiency in deciphering the intricacies of Chinese electronic medical records.