Named Entity Recognition in Electric Power Metering Domain Based on Attention Mechanism

Abstract

Named Entity Recognition (NER) is one key step for constructing power domain knowledge graph which is increasingly urgent in building smart grid. This paper proposes a new NER model called Att-CNN-BiGRU-CRF which consists of the following five layers. The prefix Att means the model is based on attention mechanism. A joint feature embedding layer combines the character embedding and word embedding based on BERT to obtain more semantic information. A convolutional attention layer combines the local attention mechanism and CNN to capture the relationship of local context. A BiGRU layer extracts higher-level features of power metering text. A global multi-head attention layer optimizes the processing of sentence level information. A CRF layer obtains the output tag sequences. This paper also constructs a corresponding power metering corpus data set with a new entity classification method. The novelties of our work are the five layer model structure and the attention mechanism. Experimental results show that the proposed model has high recall rate 88.16% and precision rate 89.33% which is better than the state-of-the-art models.