One Improved Model of Named Entity Recognition by Combining BERT and BiLSTM-CNN for Domain of Chinese Railway Construction

Abstract

There are currently few named entity recognition (NER) models in domain of Chinese railway construction. To mitigate such awkward situation, this paper uses the neural network method to sort out the basic information from Chinese text about Chinese railway construction. Concretely, this paper proposes one improved model of NER by combining bidirectional encoder representation from transformers (BERT) and convolutional long short-term memory (LSTM) network model so as to promote the NER performance of Chinese text about Chinese railway construction. Based on deep understandings of domain knowledge about Chinese railway construction, the proposed model performs targeted processing on the input, and designs a novel masking algorithm based on Chinese placenames and numbers. The proposed model further uses bidirectional LSTM (BiLSTM) network as the encoding layer, which can leverage the feature extraction capability of the convolution neural network (CNN) to improve the NER performance. Experimental results show that the F1 value of the proposed model is 7.28% higher than the traditional conditional random field (CRF) model, and the F1 value of the BERT model with mask of Chinese placenames and numbers is 3.43% higher than the original BERT model.