Federated deep long-tailed learning: A survey

Abstract

The federated learning privacy-preserving framework has achieved fruitful results in training deep models on long-tailed data across clients. This survey aims to provide a systematic overview of federated deep long-tailed learning. We analyze the problems of federated deep long-tailed learning of class imbalance/missing, different long-tailed distributions, and biased training, and summarize the current approaches that fall into the following three categories: information enhancement, model component optimization, and algorithm-based calibration. Meanwhile, we also sort out the representative open-source datasets for different tasks. We conduct abundant experiments on CIFAR-10/100-LT using LeNet-5/ResNet-8/ResNet-34 and evaluate the model performance with multiple metrics. We also consider a text classification task and evaluate the performance of multiple methods using LSTM on the 20NewsGroups-LT. We discuss the challenges posed by data heterogeneity, model heterogeneity, fairness, and security, and identify future research directions for the follow-up studies.