Subgradient Descent Learning with Over-the-Air Computation

Abstract

We consider a distributed learning problem in a communication network, consisting of N distributed nodes and a central parameter server (PS). The computation is made by the PS and is based on received data from the nodes which transmit over a multiple access channel (MAC). The objective function is a sum of the nodes’ local loss functions. This problem has attracted a growing interest in distributed sensing systems, and more recently in federated learning (FL). However, existing methods rely on the assumption that the loss functions are continuously differentiable. In this paper, we first tackle the problem when this assumption does not necessarily hold. We develop a novel algorithm, dubbed Sub-Gradient descent Multiple Access (SGMA), to solve the learning problem over MAC. In SGMA, each node transmits an analog shaped waveform of its local subgradient over MAC and the PS receives a superposition of the noisy analog signals, resulting in a bandwidth-efficient over-the-air (OTA) computation used to update the learned model. We analyze the performance of SGMA, and prove that it approaches the convergence rate of the centralized subgradient algorithm in large networks. Simulation results using real datasets demonstrate the efficiency of SGMA.