Abstract
A federated learning (FL) scheme (denoted as Fed-KSVM) is designed to train kernel support vector machines (SVMs) over multiple edge devices with low memory consumption. To decompose the training process of kernel SVM, each edge device first constructs high-dimensional random feature vectors of its local data, and then trains a local SVM model over the random feature vectors. To reduce the memory consumption on each edge device, the optimization problem of the local model is divided into several subproblems. Each subproblem only optimizes a subset of the model parameters over a block of random feature vectors with a low dimension. To achieve the same optimal solution to the original optimization problem, an incremental learning algorithm called block boosting is designed to solve these subproblems sequentially. After training of the local models, the central server constructs a global SVM model by averaging the model parameters of these local models. Fed-KSVM only increases the iterations of training the local SVM models to save the memory consumption, while the communication rounds between the edge devices and the central server are not affected. Theoretical analysis shows that the kernel SVM model trained by Fed-KSVM converges to the optimal model with a linear convergence rate. Because of such a fast convergence rate, Fed-KSVM reduces the communication cost during training by up to 99% compared with the centralized training method. The experimental results also show that Fed-KSVM reduces the memory consumption on the edge devices by nearly 90% while achieving the highest test accuracy, compared with the state-of-the-art schemes.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
More From: IEEE transactions on neural networks and learning systems
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.