Minimizing communication overhead in decentralized deep neural network (DNN) training has become a critical challenge, particularly with the increasing adoption of distributed systems for large-scale machine learning tasks. This paper introduces advanced techniques that leverage gradient compression, adaptive sparsification, and hybrid aggregation to optimize communication efficiency while maintaining model accuracy and convergence rates. Experimental results on benchmark datasets such as CIFAR-10 and ImageNet show that the proposed methods reduce communication costs by up to 70% compared to standard approaches while achieving comparable or superior model accuracy. Additionally, scalability tests on diverse neural network architectures highlight the robustness of the approach, demonstrating efficient performance across varying network sizes and computational setups. These findings underscore the potential of the proposed strategies to enable faster, cost-effective, and sustainable decentralized deep learning systems.
Read full abstract