Improved Chinese Short Text Classification Method Based on ERNIE_BiGRU Model

Abstract

This paper is aimed at traditional word embedding models and Bidirectional Encoder Representations from Transformers (BERT) that cannot learn text semantic knowledge, as well as convolutional neural network (CNN) and Bidirectional long short-term memory (BiLSTM) unable to distinguish the importance of words, proposing an improved Chinese short text classification method based on ERNIE_BiGRU model. Firstly, learning text knowledge and information through the Enhanced Representation through Knowledge Integration (ERNIE) enhances the model’s semantic representation capabilities. Secondly, considering that CNN can only extract local features of the text while ignoring the semantic relevance between contextual information, and the Bidirectional Gating Recurrent Unit (BiGRU) is simpler, has fewer network parameters and faster calculation speed than the BiLSTM, the combination of CNN and BiGRU enables the model to capture both local phrase-level features and contextual structure information. Finally, according to the importance of features, the attention mechanism is used to assign different weights to improve the classification effect of the model. The experimental results show that the ERNIE_CNN_BiGRU_Attention (ECBA) model used in this paper has achieved good results in the task of Chinese short text classification.