AdaptFL: Adaptive Federated Learning Framework for Heterogeneous Devices

Abstract

With the development of the Internet of Things (IoT), Federated Learning (FL) is extensively employed in smart cities and industrial IoT, involving numerous heterogeneous devices with varying computational and storage capabilities. Traditional FL assumes that clients have enough resources to train a unified global model from the beginning to the end of training. However, it ignores the problem of uneven and real-time changes in client resources. Additionally, there are aggregation difficulties between heterogeneous client models and global model. To address these challenges, we propose an Adaptive Federated Learning Framework for Heterogeneous Devices (AdaptFL). In AdaptFL, we employ a resource-aware neural architecture search method, which searches for models based on each client’s resource conditions. It enables AdaptFL to automatically assign customized models tailored to each client’s specific resource conditions in the current round. Additionally, we employ a staged knowledge distillation strategy to facilitate efficient distribution and aggregation between the heterogeneous global model and the client models. Experimental results demonstrate that, compared to state-of-the-art model-level heterogeneous ablation methods, AdaptFL improves global test accuracy by 4% to 15% on the SVHN dataset and enhances accuracy by 5% to 14% in scenarios with heterogeneous data. Additionally, AdaptFL effectively reduces communication overhead by over 50% across all datasets. Furthermore, it offers a degree of resilience against model poisoning attacks.