Abstract
Federated learning (FL) is a distributed model for deep learning that integrates client-server architecture, edge computing, and real-time intelligence. FL has the capability of revolutionizing machine learning (ML) but lacks in the practicality of implementation due to technological limitations, communication overhead, non-IID (independent and identically distributed) data, and privacy concerns. Training a ML model over heterogeneous non-IID data highly degrades the convergence rate and performance. The existing traditional and clustered FL algorithms exhibit two main limitations, including inefficient client training and static hyperparameter utilization. To overcome these limitations, we propose a novel hybrid algorithm, namely, genetic clustered FL (Genetic CFL), that clusters edge devices based on the training hyperparameters and genetically modifies the parameters clusterwise. Then, we introduce an algorithm that drastically increases the individual cluster accuracy by integrating the density-based clustering and genetic hyperparameter optimization. The results are bench-marked using MNIST handwritten digit dataset and the CIFAR-10 dataset. The proposed genetic CFL shows significant improvements and works well with realistic cases of non-IID and ambiguous data. An accuracy of 99.79% is observed in the MNIST dataset and 76.88% in CIFAR-10 dataset with only 10 training rounds.
Highlights
Federated learning (FL) [1, 2] has risen as a groundbreaking subdomain of machine learning (ML) that enables Internet of ings (IoT) devices to contribute their real-time data and processing to train ML models
E limited feasibility of ML in industrial and IoT applications is overturned by the introduction of FL
We propose three novel contributions to lessen the empirical risk in FL, as shown in Figure 1: (i) Clustering of clients solely based on model hyperparameters to increase the learning efficiency per unit training of the model (ii) Implementation of density-based clustering, i.e., DBSCAN, on the hyperparameters for proper analysis of devices properties (iii) Introduction of genetic evolution of hyperparameters per cluster for finer tuning of individual device models and better aggregation
Summary
Federated learning (FL) [1, 2] has risen as a groundbreaking subdomain of machine learning (ML) that enables Internet of ings (IoT) devices to contribute their real-time data and processing to train ML models. Computational power and properties of data (ambiguity, size, and complexity) vary drastically, and diversely trained client models are hard to aggregate. In a realistic scenario of thousands of edge devices, the updated global model may not converge at all Existing aggregating algorithms such as FedAvg and FedMA [14] focus more on integration of weights of the local models. We introduce a new algorithm, namely, Genetic CFL, that clusters hyperparameters of a model to drastically increase the adaptability of FL in realistic environments. Hyperparameters such as batch size and learning rate are core features of any MFL model.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
