Multi-Channel Text Classification Model Based on ERNIE

Abstract

Aiming at the large amount of news and review text data, sparse features, and the inability of traditional text feature representation to dynamically obtain grammatical structure, semantic information, and multi-dimensional rich feature representation of entity phrases. This paper proposes to obtain more generalized knowledge semantic feature information such as rich context phrases, entity words and so on by integrating knowledge enhanced semantic representation (Enhanced Representation Through Knowledge Integration, ERNIE). The pre-trained language model ERNIE hides words and entities by random Semantic unit prediction context realizes word vector language representation, and the output vector representation of ERNIE is input to BiLSTM, Attention mechanism and DPCNN network model to generate high-order text feature vectors, and each channel vector is processed by BatchNormalization and ReLU activation functions respectively.Thus, the semantic description information of the multi-channel word vector is fused. The model proposed in this paper can not only improve the training speed and prevent overfitting, but also enhance the feature information such as semantics and grammatical structure, thereby improving the text classification effect. By comparing the two datasets with other improved ERNIE models in terms of accuracy, precision, recall, and F1 value, the experimental results show that the model proposed in this paper can obtain multi-dimensional rich semantic grammatical structure features for text classification, and then improve the Text classification effect.