FedACQ: adaptive clustering quantization of model parameters in federated learning

Abstract

PurposeFor privacy protection, federated learning based on data separation allows machine learning models to be trained on remote devices or in isolated data devices. However, due to the limited resources such as bandwidth and power of local devices, communication in federated learning can be much slower than in local computing. This study aims to improve communication efficiency by reducing the number of communication rounds and the size of information transmitted in each round.Design/methodology/approachThis paper allows each user node to perform multiple local trainings, then upload the local model parameters to a central server. The central server updates the global model parameters by weighted averaging the parameter information. Based on this aggregation, user nodes first cluster the parameter information to be uploaded and then replace each value with the mean value of its cluster. Considering the asymmetry of the federated learning framework, adaptively select the optimal number of clusters required to compress the model information.FindingsWhile maintaining the loss convergence rate similar to that of federated averaging, the test accuracy did not decrease significantly.Originality/valueBy compressing uplink traffic, the work can improve communication efficiency on dynamic networks with limited resources.