Label Oriented Hierarchical Attention Neural Network for Short Text Classification

Abstract

Listen

Translate article

Text classification task is a very common task in natural language processing. The conventional text classification task model often uses word bag model or representation model, but the existing model usually deals with long text classification task, which is not suitable for short text classification. Short text features are relatively fewer and often need more sophisticated feature extraction, and the role of keywords play great importance roles in the classification of short text. In this paper, we propose a label-oriented hierarchical attention mechanism network for short text classification. The model achieves better results on public data sets of Tiao and Weibo, compared with convolutional neural network CNN, GATE control unit neural network GRU, gate control unit neural network fusion convolutional neural network GRU-CNN and translation model Transfomer. It is proved that the model has good performance. Our model has two significant advantages: (1) it is a hierarchical structure consisting of two levels of attention mechanisms, which facilitates interpretation and analysis; (2) Compared with other architectures, this model can be used to extend multi-label text classification tasks. In addition, it can be used for long-term importance analysis in many industrial scenarios.