Adaptive Hierarchical Federated Learning Over Wireless Networks

Abstract

Federated learning (FL) is promising in enabling large-scale model training by massive devices without exposing their local datasets. However, due to limited wireless resources, traditional cloud-based FL system suffers from the bottleneck of communication overhead in core network. Fueled by this issue, we consider a hierarchical FL system and formulate a joint problem of edge aggregation interval control and resource allocation to minimize the weighted sum of training loss and training latency. To quantify the learning performance, an upper bound of the average global gradient deviation, in terms of the edge aggregation interval, the training latency, and the number of successfully participating devices, is derived. Then an alternative problem is formulated, which can be decoupled into an edge aggregation interval control problem and a resource allocation problem, and solved by an iterative optimization algorithm. Specifically, given the resource allocation strategy, a relaxation and rounding method is proposed to optimize the edge aggregation interval. The problem of resource allocation including training time allocation and bandwidth allocation is solved separately based on the convex optimization theory. Simulation results show that the proposed algorithm, compared to the benchmarks, can achieve higher learning performance with lower training latency, and is capable of adaptively adjusting the edge aggregation interval and the resource allocation strategy according to the training process.