AFL-DCS: An asynchronous federated learning framework with dynamic client scheduling

Abstract

The emerging federated learning is a distributed machine learning paradigm which enables training a global model on a massive number of edge devices while protecting the privacy of local data. In the typical federated learning paradigm, the global model is updated with a synchronized protocol, which requires the server to wait for all clients to return their model parameters before updating the global model in each round. However, the straggler effect due to heterogeneous devices may cause a serious degradation in the training efficiency of synchronous federated learning. Asynchronous federated learning can effectively alleviate the inefficiency of training caused by heterogeneous devices, but the asynchronous update protocol makes the global model more vulnerable to heterogeneous data. The global model of asynchronous federated learning may be difficult to converge or even not under non-IID settings. In this paper, we propose an asynchronous federated learning framework with dynamic client scheduling (AFL-DCS) to mitigate the negative impact of data heterogeneity and dynamic clients. By dynamically clustering clients, AFL-DCS can effectively handle the dynamic check-in clients and their non-IID data. We evaluate AFL-DCS on three datasets (MNIST, FashionMNIST and CIFAR-10) under several different types of non-IID settings and compare AFL-DCS with typical asynchronous federated learning frameworks. Experimental results show that AFL-DCS can reduce the number of communication rounds by up to 39.0%, 52.4% and 89.6% on average, and reduce the accuracy variance by up to 88.9%, 80.0% and 76.8% on average, respectively, which confirms that our framework can significantly enhance the learning efficiency, model performance and stability of asynchronous federated learning under various non-IID settings.