Learning-driven Spatio-temporal Feature Extraction for Violence Detection in IoT Environments

Abstract

With the increasing growth of security concerns, the usage of surveillance cameras is also extending rapidly. Thus, the research on automatic violence detection from this surveillance footage is compelling. On the other hand, deep learning (DL) models have achieved satisfying accuracy in violence detection. However, DL models suffer from higher computational cost with a larger memory footprint. Therefore, such models become inappropriate in real-world applications, such as Internet of Things (IoT)-based systems. The architectures of existing approaches are also inefficient for spatial and temporal feature extractions. Considering these challenges, this article presents a novel architecture for violence detection, named Dual Spatio-temporal Convolutional Network (DSTCN). The proposed model extracts temporal and spatial information by 1D, 2D, and 3D convolutional neural networks (CNNs) from RGB frames. Though this model combines multi-dimensional CNNs, it remains lightweight. Due to the low channel capacity, the model learns to extract meaningful spatial and temporal information. Through simulation results, we demonstrate that our model achieves the accuracy of 86.0% on the largest violence detection benchmark dataset while keeping the low complexity. As a result, maintaining a balanced mix of accuracy, memory usage, and processing time, the proposed system becomes ideal for IoT-based surveillance systems.