A Memory-Efficient Federated Kernel Support Vector Machine for Edge Devices.

Abstract

A federated learning (FL) scheme (denoted as Fed-KSVM) is designed to train kernel support vector machines (SVMs) over multiple edge devices with low memory consumption. To decompose the training process of kernel SVM, each edge device first constructs high-dimensional random feature vectors of its local data, and then trains a local SVM model over the random feature vectors. To reduce the memory consumption on each edge device, the optimization problem of the local model is divided into several subproblems. Each subproblem only optimizes a subset of the model parameters over a block of random feature vectors with a low dimension. To achieve the same optimal solution to the original optimization problem, an incremental learning algorithm called block boosting is designed to solve these subproblems sequentially. After training of the local models, the central server constructs a global SVM model by averaging the model parameters of these local models. Fed-KSVM only increases the iterations of training the local SVM models to save the memory consumption, while the communication rounds between the edge devices and the central server are not affected. Theoretical analysis shows that the kernel SVM model trained by Fed-KSVM converges to the optimal model with a linear convergence rate. Because of such a fast convergence rate, Fed-KSVM reduces the communication cost during training by up to 99% compared with the centralized training method. The experimental results also show that Fed-KSVM reduces the memory consumption on the edge devices by nearly 90% while achieving the highest test accuracy, compared with the state-of-the-art schemes.