GFL: Federated Learning on Non-IID Data via Privacy-Preserving Synthetic Data

Abstract

Federated learning (FL) enables large amounts of participants to construct a global learning model, while storing training data privately at local client devices. A fundamental issue in FL systems is the susceptibility to the highly skewed distributed data. A series of methods have been proposed to mitigate the Non-IID problem by limiting the distances between local models and the global model, but they cannot address the root cause of skewed data distribution eventually. Some methods share extra samples from the server to clients, which requires comprehensive data collection by the server and may raise potential privacy risks. In this work, we propose an efficient and adaptive framework, named Generative Federated Learning (GFL), to solve the skewed data problem in FL systems in a privacy-friendly way. We introduce Generative Adversarial Networks (GAN) into FL to generate synthetic data, which can be used by the server to balance data distributions. To keep the distribution and membership of clients' data private, the synthetic samples are generated with random distributions and protected by a differential privacy mechanism. The results show that GFL significantly outperforms existing approaches in terms of achieving more accurate global models (e.g., 17% - 50% higher accuracy) as well as building global models with faster convergence speed without increasing much computation or communication costs.