Towards asynchronous federated learning based threat detection: A DC-Adam approach

Abstract

The increasing popularity and widespread use of Internet of Things (IoT) and Cyber-Physical Systems (CPS) technologies have produced a significant need for the integration of cloud and edge computing with distributed detection solutions to handle the growing volume of distributed security threats. While deep learning-based approaches have been used to detect anomalous behaviors in complex data patterns, the heterogeneity in IoT networks still poses paramount challenges to update synchronization across learning nodes in distributed training. Particularly, the non-independent and identically distributed (non-IID) data patterns over remote nodes significantly affect the performance of model training provisioned on cloud and edge computing servers, and most existing works have assumed a homogeneous setting. The heterogeneity brings the gradient delay problem, causing the gradient inconsistency in the barrier-free asynchronous mode. In this paper, we propose a Delay Compensated Adam (DC-Adam) approach, an asynchronous federated learning-based detection approach, for IoT devices with limited resources. To overcome the notorious gradient delay problem, we develop a Taylor Expansion-based scheme to compensate for the inconsistency caused by asynchronous communication. Moreover, a pre-shared data training strategy for non-IID data is developed to avoid the convergence divergence under the non-IID patterns. After the collaborative model training procedure, we append an additional local training process at each client to fit respective patterns. Via a combination of theoretical analysis of convergence and practical experimental results, we validate the efficacy of our proposed approach compared to the other state-of-the-art approaches. Compared with benchmark approaches, we demonstrate that our proposed method can converge stably, and that it outperforms the barrier-free asynchronous federated learning by 12.8% (accuracy), 14% (precision). 8.71% (recall), and 11.16% (F1 score) on average.