CDGAN-BERT: Adversarial constraint and diversity discriminator for semi-supervised text classification

Abstract

Semi-supervised generative adversarial learning (SS-GAN) significantly improves the model’s performance with limited labeled data, especially on text classification. However, these SS-GAN-like works pay more attention to the more real properties of the generated samples and ignore their adversarial nature, which may prevent the model from learning more higher-quality data representations. And the existing single discriminator may produce discrimination ambiguity when discriminating against tasks with different discriminative properties (simultaneously identifying real or fake and classifying categories), which may harm the model’s classification performance. In this paper, we propose a novel CDGAN-BERT, an SS-GAN-based architecture with the adversarial constraint and a diversity discriminator. Specifically, CDGAN-BERT first focuses on adversarial constraint, which calculates the model’s intrinsic state representation with the reciprocal of Mean Squared Error (MSE), further keeping adversarial of the generated samples relative to the real data distribution. Then, a diversity discriminator is designed to improve the model’s classification performance by alleviating discriminative ambiguity, which refines tasks with different discriminative attributes by adding an additional authenticity discriminator. We validate our model on 6 text datasets and achieve significant improvements, especially with limited supervision. The experimental results show that our method outperforms or gets comparable results to other state-of-the-art semi-supervised learning methods on several datasets. Especially for the datasets of QC-Fine and QC-Coarse with limited labeled data, our CDGAN-BERT achieves the best average Micro-F1 scores of 72.5% and 92.037%, respectively.