Two-stage sampling with predicted distribution changes in federated semi-supervised learning

Abstract

Federated semi-supervised learning (FSSL) involves training a model in a federated environment using a few labeled samples and many unlabeled samples. Compared with semi-supervised learning, FSSL faces more complex data situations, especially when data are non-independently and identically distributed (non-IID), adding more challenges to the learning process. The previous method addresses the aforementioned issues by enlarging the training sample space through multiple client random sampling and reweighting the parameters. Although it achieves high accuracy, it sacrifices communication efficiency. In this study, we propose PDCFed, a two-stage sampling method that uses the Predicted Distribution Changes of samples after different data augmentations. We evaluate the credibility of the samples based on the maximum probability predicted by weak augmentation. When the samples are in a less reliable space, they are further sampled after adjusting the predicted distribution changes using a Gaussian function. To enhance the model’s generalization ability, an entropy penalty term is incorporated after unsupervised training loss. Extensive experiments demonstrate that this method outperforms existing methods on three datasets with non-IID data and significantly improves communication efficiency.