Label-text bi-attention capsule networks model for multi-label text classification

Abstract

Multi-label text classification (MLTC) is the process of establishing relationships between documents and their corresponding labels. Previous research has focused on mining textual information, treating labels as information-less vectors in classification. This ignores the semantic and dependency relationships of labels. In real-life scenarios, the neglect of label information contradicts the classification process, which presents significant challenges for MLTC tasks. Label embedding partially resolves label information loss. Efficiently exploring semantic and dependency relationships of labels and their text connections remains a new challenge. In this paper, we propose a Label-Text Bi-Attention Capsule Networks (LTBACN) model for in-depth exploration of the dependency relationships between labels and text. Specifically, we first incorporate label information into nodes through label embedding, construct a graph structure to represent the dependency relationships between labels, and use Graph Convolutional Networks (GCN) to propagate information between nodes to further mine the relationships between labels. Subsequently, we employ a label-text bi-attention mechanism to learn the feature relationships between labels and text. The label-to-text attention mechanism extracts label-relevant text representations, while the text-to-label attention mechanism extracts the most relevant label representations for the text. We then merge these two types of feature representations to obtain fused representations that incorporate label-text bi-directional information. Finally, the fused features are fed into a capsule network classifier to capture multi-level semantic information and match the corresponding labels. The experimental results demonstrate that LTBACN outperforms other methods in terms of classification effectiveness. Compared to state-of-the-art methods, LTBACN achieves a significant improvement of 0.41%–0.68% in Micro−F1 measure, 0.52%–3.26% in Macro−F1 measure, 0.32%–2.18% in P@k measure, and 0.01%–1.18% in nDCG@k measure on the AAPD and RCV1-v2 datasets.