Contrastive classification: A label-independent generalization model for text classification

Abstract

Although existing text classification methods have achieved SOTA results on most tasks, their generalization ability is inferior to their excellent performance on a single task. For this issue, existing research mainly improves the generalization of text classification models by adding label semantics to model training and allowing models to see label semantic information. However, its generalization ability is still limited by the description and quantity of labels. This paper presents a Contrastive Classification method based on contrastive learning to enhance the generalization of text classification models. By transforming all text classification tasks into semantic understanding tasks, the method strengthens the correlation between pre-training tasks and downstream tasks, better preserving the generalization of pre-trained models. At the same time, it unifies the training objectives of all text classification tasks, reducing the differences between different tasks. We verified the model’s generalization ability on 17 different zero-shot tasks, verified the model’s cross-domain generalization ability and growth potential on 3 question–answer and 2 reading comprehension tasks, and verified the model’s performance on 12 supervised text categorization tasks. Experimental results show that Contrastive Classification frees text classification models from the limitations of classification labels and improves them in different dimensions: (1) Robustness, (2) Cross-domain generalization ability, and (3) Growth potential. We plan to release our code at https://github.com/liangyi-qianwan/Contrastive-classification-A-label-independent-generalization-model-for-text-classification/tree/main.