Hierarchical Recurrent and Convolutional Neural Network Based on Attention for Chinese Document Classification

Abstract

Document classification is one of the foundational tasks of Natural Language Processing (NLP) applications. In this paper, we introduce an attention based hierarchical recurrent and convolutional neural network to do Chinese document classification. In our model, we not only apply a recurrent neural network to capture information of each sentence as far as possible when learning word representations, but also apply a convolutional neural network to get the contextual information of the whole document. Besides, we also employ an attention layer that automatically judges which words play key roles in each sentence to capture the key components in sentences. We conduct experiments on two commonly used Chinese document datasets. The experimental results show that our proposed method outperforms the baseline methods on these datasets in several indicators.