Communication-efficient federated learning via personalized filter pruning

Abstract

With the popularity of mobile devices and the continuous growth of interactive data, FL (Federated Learning) has gradually become an effective mean to address the problems of privacy leakage and data silos. However, due to the heterogeneity and imbalance of participants, FL faces many challenges, including model accuracy, security, heterogeneous devices and data, privacy preservation, as well as communication overhead and efficiency. To address challenges such as high communication overhead and low model accuracy in FL, we innovatively introduce model pruning into the FL framework and propose a personalized filter pruning-based FL method named PF2 Learning (Personalized Filter Pruning Federal Learning). This method achieves reasonable filter pruning for all local models by performing personalized pruning on each local model. Specifically, on the device side, we use a pruning strategy based on the norm geometric median, which also considers the order dependence between adjacent layers and evaluates the contribution of filters involved in pruning to optimize the pruning for local models at a unified pruning rate. On the server side, we calculate the unified pruning ratio of the model based on the contribution of participants' model filters and training errors to maintain the consistency of the participants' model structures. To validate the effectiveness of our proposed method, we conducted federated image classification tasks on ResNet-18, VGG-11, DenseNet-121, and InceptionNet-V1 models using CIFAR-10, FEMNIST, and ImageNet datasets. The experimental results show that PF2 Learning outperforms most FL pruning methods and exhibits better model performance and accuracy.