Using Synthetic Data to Reduce Model Convergence Time in Federated Learning

Abstract

Federated Learning (FL) is a hot new topic in collaborative training of machine learning problems. It is a privacy-preserving distributed machine learning approach, allowing multiple clients to jointly train a global model under the coordination of a central server, while keeping their sensitive data private. The problem with FL systems is that they require intense communication between the server and clients to achieve the final machine learning model. Such complexity increases with the number of clients participating and the complexity of the model sought. In this paper, we introduce synthetic data generation into FL systems with the intention of reducing the number of iterations required for model convergence. In this novel method, clients generate synthetic datasets modeling their private data. The synthetic datasets are then sent to the central server and are used to generate a cognizant initial model. Our experiments show that such conscious method for generating the initial model lowers the number of iterations by a factor of more than 4 without affecting the model accuracy. As such it enhances the overall efficiency of FL systems.