Analog Gradient Aggregation for Federated Learning Over Wireless Networks: Customized Design and Convergence Analysis

Abstract

This article investigates the analog gradient aggregation (AGA) solution to overcome the communication bottleneck for wireless federated learning applications by exploiting the idea of analog over-the-air transmission. Despite the various advantages, this special transmission solution also brings new challenges to both transceiver design and learning algorithm design due to the nonstationary local gradients and the time-varying wireless channels in different communication rounds. To address these issues, we propose a novel design of both the transceiver and learning algorithm for the AGA solution. In particular, the parameters in the transceiver are optimized with the consideration of the nonstationarity in the local gradients based on a simple feedback variable. Moreover, a novel learning rate design is proposed for the stochastic gradient descent algorithm, which is adaptive to the quality of the gradient estimation. Theoretical analyses are provided on the convergence rate of the proposed AGA solution. Finally, the effectiveness of the proposed solution is confirmed by two separate experiments based on linear regression and the shallow neural network. The simulation results verify that the proposed solution outperforms various state-of-the-art baseline schemes with a much faster convergence speed.