TDS-Net: Transformer enhanced dual-stream network for video Anomaly Detection

Abstract

Video anomaly detection is a critical research area, driven by the increasing reliance on surveillance systems to maintain public safety and security. The implementation of surveillance cameras in public spaces is driven by the specific goal of monitoring incidents involving theft, vandalism, traffic accidents, etc. However, the reliance on human oversight for anomaly detection introduces a susceptibility to errors, emphasizing the urgent need for the development of efficient algorithms capable of autonomously detecting anomalies in video footage. Furthermore, anomalous patterns within video data often have a complex background, unexpected events, variations in the scale of the objects, and anomalous appearances, which makes their detection challenging. Therefore we introduce a novel Dual Stream Transformer Network (TDS-Net) to address these challenges. TDS-Net concurrently extracts RGB and flow features, using a transformer network for sequential pattern learning. The parallel extraction of RGB features allows the model to learn patterns, while the dedicated flow module adeptly handles motion features. The significance of this innovative network for video anomaly detection is that it enhances the ability to identify and analyze unusual patterns or events within video data. Furthermore, the TDS-Net has demonstrated superior accuracy compared to baseline methods across benchmark datasets, including ShanghaiTech, CUHK Avenue, and UCF Crime. We also carried out an ablation study, offering insights into the individual contributions of the components of the TDS-Net architecture.