Hierarchical text classification with multi-label contrastive learning and KNN

Abstract

Listen

Translate article

Given the complicated label hierarchy, hierarchical text classification (HTC) has emerged as a challenging subtask in the realm of multi-label text classification. Existing methods enhance the quality of text representations by contrastive learning, but this supervised contrastive learning is designed for single-label setting and has two main limitations. On one hand, sample pairs with completely identical labels which should be treated as positive pairs are ignored. On the other hand, a simple pair is deemed as an absolutely positive or negative pair, which lacks consideration about the situation where sample pairs share some labels while having labels unique to each sample. Therefore, we propose a method combining multi-label contrastive learning with KNN (MLCL-KNN) for HTC. The proposed multi-label contrastive learning method can make text representations of sample pairs having more shared labels closer and separate those with no labels in common. During inference, we employ KNN to retrieve several neighbor samples and regard their labels as additional prediction, which is interpolated into the model output to further improve the performance of MLCL-KNN. Compared with the strongest baseline, MLCL-KNN achieves average improvements of 0.31%, 0.76%, 0.83%, and 0.43% on Micro-F1, Macro-F1, accuracy, and HiF respectively, which demonstrates its effectiveness.