Byzantine-robust federated learning via cosine similarity aggregation

Abstract

Federated Learning (FL) is proposed to train a machine learning model for clients with different training data. During the training of FL, a centralized server is usually employed to aggregate local models from clients iteratively. The aggregation process suffers from Byzantine attacks, where clients’ models could be maliciously modified by attackers to degrade the training performance. Existing defense aggregation solutions use distances or angles between different gradients to identify and eliminate malicious models from clients. However, they do not work well due to the high dimensional property of the machine learning model. Distance-based solutions cannot effectively identify attackers when the gradient direction of the model is maliciously tampered with. Angle-based solutions face the issue of low model accuracy for large models. In this paper, we propose Convolutional Kernel Angle-based Defense Aggregation (CKADA) to improve defense performance under various Byzantine attacks. The key of CKADA is to use the angle between convolutional kernels as the attack detection metric because the obtuse angle indicates the wrong training direction. CKADA calculates the angle between a client’s convolutional kernel gradients and the server’s convolutional kernel gradients as the attacker detection metric and eliminates convolutional kernel gradients of clients that create an obtuse angle to mitigate the impact of attackers on the model. We evaluate the performance of CKADA using AlexNet and ResNet-50 under two typical attacks. Simulation results show that CKADA mitigates the impact of Byzantine attacks and outperforms existing angle-based solutions and distance-based solutions by improving inference accuracy up to 67% and 88% respectively.