Improved sequence generation model for multi-label classification via CNN and initialized fully connection

Abstract

In multi-label text classification, considering the correlation between labels is an important yet challenging task due to the combination possibility in the label space increasing exponentially. In recent years, neural network models have been widely applied and gradually achieved satisfactory performance in this field. However, existing methods either not model the fully internal correlations among labels or not capture the local and global semantic information of text simultaneously, which somewhat affects the classification results finally. In this paper, we implement a novel model for multi-label classification based on sequence-to-sequence learning, in which two different neural network modules are employed, named encoder and decoder respectively. The encoder uses the convolutional neural network to extract the high-level local sequential semantic, which is combined with the word vector to generate the final text representation through the recurrent neuron network and attention mechanism. The decoder, besides using a recurrent neural network to capture the global label correlation, employs an initialized fully connection layer to capture the correlation between any two different labels. When trained on RCV1-v2, AAPD and Ren-CECps datasets, the proposed model outperforms previous work in main evaluation metrics of hamming loss and micro-F1 score.