A Study on the EDA-based Classification of News Text

Abstract

At present, the commonly used word vector methods for news text classification mostly adopt Word2Vec, Glove, Bert and other word vector models, ignoring the remote context connections of Chinese text itself. TextCNN, RNN, BiLSTM and other neural network classification models lack the extraction of important information features of the text, and the word dependence inside the text is not strong, resulting in inaccurate classification results. To solve the above problems, this paper proposes the DPCNN-Attention news text classification model based on ERNIE’s pre-training model(EDA for short, the following general). The DPCNN neural network model with Mish() activation function is adopted to obtain the maximum length of semantic association between long distance texts in news texts. By adding attention mechanism into EDA model, in the feature extraction process, according to the importance of words to the classification results, different weights are assigned to them to enhance the word dependence relationship within the text, thus greatly improving the classification accuracy. The EDA model was experimentally verified on the THUCNews dataset. The results showed that the EDA model improved by about 6% compared with BERT’s pre-training model, and 0.4% compared with ERNIE’s pre-training model. The loss rate decreased by 0.2 compared with BERT and 0.01 compared with ERNIE’s.