CluFL: Cluster-driven Weighted FL Model Aggregation Strategy

Abstract

Federated learning (FL) has become a promising machine learning (ML) paradigm for training machine learning models over distributed datasets, owing to its low communication costs and privacy preserving property. To date, the most commonly adopted model fusion mechanism in FL is average aggregation. However, it has been shown that this average aggregation mechanism performs poorly in heterogeneous systems, especially for non-independent and identically distributed (NonIID) data. In order to address this challenge, we propose a weighted FL model aggregation strategy for each client based on clustering, termed CluFL. Specifically, CluFL first measures the similarities among uploaded models from clients through their parameters using a spectral clustering algorithm. Then, CluFL assigns aggregation weights according to the similarity of the intra-cluster global model for each cluster and the average model across the clusters. Further, we derive a convergence bound on the CluFL algorithm considering a practical nonconvex setting of neural network training. This bound reveals that the proposed CluFL algorithm can achieve a convergence speed in the order of O(1/T). Extensive experiments have been conducted on both FashionMNIST and CIFAR-10 datasets and show that CluFL outperforms the state-of-the-art FL algorithms in terms of accuracy and communication efficiency.