Communication-Efficient Federated Learning with Adaptive Consensus ADMM

Abstract

This paper proposes utilizing federated learning (FL), a distributed learning paradigm, to process large, decentralized, and heterogeneous edge data in the context of Internet of Things (IoT) devices. However, heterogeneity and high communication costs are two primary challenges that hinder the efficacy of federated learning. To overcome these challenges, we have designed an algorithm, FedACADMM, which applies the adaptive consensus alternating direction method of multipliers (ACADMM) to federated learning clients (i.e., the edge mobile devices) to tackle the heterogeneity problem in federated networks. Importantly, the cost per round of communication for FedACADMM remains consistent with FedAvg and FedProx without adding any extra workload. Furthermore, our experimental results demonstrate that FedACADMM outperforms baseline methods with a realistic set of federated datasets, displaying enhanced convergence robustness. Notably, in highly heterogeneous scenarios, FedACADMM exhibits significant stability and rapid convergence, requiring at least 75% fewer communication rounds to achieve the target accuracy.