Cooperative abnormal sound event detection in end-edge-cloud orchestrated systems

Abstract

In this paper, we propose a novel cooperative abnormal sound event detection framework for city surveillance in end-edge-cloud orchestrated systems. A novel offloading decision-making scheme that leverages hierarchical computational capabilities is proposed to speed up the detection process. The audio pre-processing (feature extraction) and post-processing (sound source localization and sound event classification) can be locally executed or offloaded to the edge or cloud based on the calculation of the so-called communication-to-computation ratio. Furthermore, considering the biased audio information due to source-sensor geometries, a cooperative decision-making algorithm is proposed to aggregate the sound event classification results with adaptive control and ensemble learning. In the audio pre-processing, the log-mel spectrogram and time of arrival information are first extracted from the audio waveform captured by the distributed acoustic sensors and then sent to the computation entity assigned by the offloading scheme. In the audio post-processing, the sound source is localized through least-square minimization. Guided by the localized sound source, the spectrograms are fed into the pre-trained neural networks and then the result aggregation algorithm for further classification. Extensive experiments regarding latency and detection accuracy show the superiority and robustness of the proposed scheme, avoiding the cumulative latency caused by the increased number of sensors while maintaining high detection accuracy.