Multi Text Classification Model Based on BRET-CNN-BiLSTM

Abstract

When the Bert pre-trained model is used for the Chinese text classification task, the internal parameters of the model are relatively fixed. In the process of training smaller data sets, it is easy to cause over-fitting phenomenon. Therefore, a multiple model structure based on Bert-CNN-BiLSTM is proposed. In this structure, the Bert model is used as the text information extractor, after the output features of Bert model, multi head attention and TEXT-CNN model are used for further feature information extraction, low-dimensional feature vectors with more dense semantic information are generated and spliced to improve the information entropy of text vectors. Finally, the BiLSTM with self-attention is used to extract the information of different words in text information and then classify the text information, and the loss function with Flooding mechanism is used to carry out back propagation to further prevent the over-fitting problem of the model on smaller data sets and enhance the generalization ability of the model. Compared with the traditional Bert, LSTM, TEXT-CNN models, the accuracy, precision, recall and F1 measure of this model are all better than those of traditional Bert, LSTM and TEXT-CNN models. Experiments show that the model can effectively extract the feature information from the text and improve the accuracy of the text classification task.