A dynamic adaptive iterative clustered federated learning scheme

Abstract

Clustered federated learning (CFL), as an important research branch of personalized federated learning (FL), can better cope with the highly statistically heterogeneous federated learning environment and provide higher quality services to clients. However, existing CFL schemes have difficulties in adapting to real-time data distribution changes due to disadvantages such as relatively fixed cluster structure. This poses a great challenge to the practical deployment of CFL schemes. To address the common problems of existing CFL schemes, we propose a more flexible dynamic adaptive cluster federated learning scheme (AICFL). AICFL uses the mutual sensitivity between models and data as intuition to perform cluster identity estimation, cluster addition, cluster model updating, and cluster deletion in the early iterations of FL to find the optimal client cluster partitioning. Firstly, this process does not require a priori estimation of the number of clusters and does not require the online participation of all clients. Secondly, during cluster partitioning, AICFL is able to adjust the cluster structure in real time based on the overall data distribution. Moreover, AICFL has the same ability to adapt to changes in the system environment in the middle and late stages of FL. The experimental results show that our scheme gives the most reasonable cluster partitioning results in all cases which indicates that AICFL is able to cope with the above-mentioned distribution changes well, and has better adaptability and better flexibility than other schemes.