Fourier Enhanced MLP with Adaptive Model Pruning for Efficient Federated Recommendation

Abstract

AbstractFederated learning (FL) is gradually gaining traction as the de facto standard for distributed recommendation model training that takes advantage of on-device user data while reducing server costs. However, the computation resources of user devices in FL are usually much more limited compared to servers in a datacenter, which hinders the application of some advanced recommendation models (e.g., Transformer-based models) in FL. In addition, models with better recommendation performance tend to have more parameters, which increases the cost of communication between servers and user devices. Therefore, it is difficult for existing federated recommendation methods to achieve a good trade-off between recommendation accuracy and computation and communication costs. As a response, we propose a novel federated recommendation framework for efficient recommendations. First, we propose an all-MLP model by replacing the self-attention sublayer in a Transformer encoder with a Fourier sublayer, in which the noise information in the user interaction data is effectively attenuated using Fast Fourier Transform and learnable filters. Second, we adopt an adaptive model pruning technique in the FL framework, which can significantly reduce the model size without affecting the recommendation performance. Extensive experiments on four real-world datasets demonstrate that our method outperforms existing federated recommendation methods and strikes a good trade-off between recommendation performance and model size.KeywordsRecommender systemSequential recommendationFederated recommendationFederated learningAll-MLP model