Supervised Contrastive Learning with Term Weighting for Improving Chinese Text Classification

Abstract

With the rapid growth of information retrieval technology, Chinese text classification, which is the basis of information content security, has become a widely discussed topic. In view of the huge difference compared with English, Chinese text task is more complex in semantic information representations. However, most existing Chinese text classification approaches typically regard feature representation and feature selection as the key points, but fail to take into account the learning strategy that adapts to the task. Besides, these approaches compress the Chinese word into a representation vector, without considering the distribution of the term among the categories of interest. In order to improve the effect of Chinese text classification, a unified method, called Supervised Contrastive Learning with Term Weighting (SCL-TW), is proposed in this paper. Supervised contrastive learning makes full use of a large amount of unlabeled data to improve model stability. In SCL-TW, we calculate the score of term weighting to optimize the process of data augmentation of Chinese text. Subsequently, the transformed features are fed into a temporal convolution network to conduct feature representation. Experimental verifications are conducted on two Chinese benchmark datasets. The results demonstrate that SCL-TW outperforms other advanced Chinese text classification approaches by an amazing margin.