A Text Category Detection and Information Extraction Algorithm with Deep Learning

Abstract

In order to solve the problem that the text classification model based on neural network is easy to over-fit and ignore the key words in sentences in the training process, a Bi-GRU Chinese text classification model based on hierarchical Attention mechanism is proposed. The model introduces the idea of layering, uses bi-directional gated cyclic neural network to learn the text representation at word level and sentence level, uses Self-Attention hierarchical model to obtain the information of the influence of words and sentences on text classification, shares the weight between embedded layer and softmax layer by binding, and uses AMSBound optimization method to obtain the optimal weight matrix quickly and effectively while reducing the parameters in the model. Two commonly used Chinese data sets, FudanSet and THUCNews, are tested on the long Chinese text classification data set FudanSet. The experimental results show that the accuracy, recall rate and F-score of this model are better than Text-CNN model, Attention-BiLSTM model and Bi-GRU_CNN model, and the accuracy, recall rate and F-score index are improved by 5.9%, 5.8% and 4.6%, respectively.