ADC-DL: Communication-efficient distributed learning with hierarchical clustering and adaptive dataset condensation

Abstract

The rapid growth of modern mobile devices leads to a large number of distributed data, which is extremely valuable for learning models. Unfortunately, model training by collecting all these original data to a centralized cloud server is not applicable due to data privacy and communication costs concerns, hindering artificial intelligence from empowering mobile devices. Moreover, these data are not identically and independently distributed (Non-IID) caused by their different context, which will deteriorate the performance of the model. To address these issues, we propose a novel Distributed Learning algorithm based on hierarchical clustering and Adaptive Dataset Condensation, named ADC-DL, which learns a shared model by collecting the synthetic samples generated on each device. To tackle the heterogeneity of data distribution, we propose an entropy topsis comprehensive tiering model for hierarchical clustering, which distinguishes clients in terms of their data characteristics. Subsequently, synthetic dummy samples are generated based on the hierarchical structure utilizing adaptive dataset condensation. The procedure of dataset condensation can be adjusted adaptively according to the tier of the client. Extensive experiments demonstrate that the performance of our ADC-DL is more outstanding in prediction accuracy and communication costs compared with existing algorithms.