Symmetric ADMM-Based Federated Learning with a Relaxed Step

Abstract

Federated learning facilitates the training of global models in a distributed manner without requiring the sharing of raw data. This paper introduces two novel symmetric Alternating Direction Method of Multipliers (ADMM) algorithms for federated learning, Two algorithm utilize a convex combination of current local and global variables to generate relaxed steps to improve computational efficiency. They also integrate two dual-update steps with varying relaxation factors into the ADMM framework to boost the accuracy and the convergence rate. Another key feature is the use of weak parametric assumptions to enhance its computational feasibility. Furthermore, the global update in the second algorithm occurs only at certain steps (e.g., at steps being a multiple of a pre-defined integer) to improve the communication efficiency. Theoretical analysis demonstrates linear convergence under reasonable conditions, and experimental results confirm the superior convergence and heightened efficiency of the proposed algorithms compared to the existing methodologies.