Fast Violence Recognition in Video Surveillance by Integrating Object Detection and Conv-LSTM

Abstract

Video surveillance involves petabytes of data storage requiring expensive hardware, which might also be time-inefficient. The aim of this article is, therefore, to develop an intelligent system capable of analyzing long sequences of videos captured from CCTV, helping to mitigate catastrophe and mitigate the violent threats faced by citizens every day, economically and efficiently. Existing models have achieved high accuracy on available datasets, the primary focus is to improve speed (time-efficient) of violence detection and use very little storage (economical) such that the system can be used in real-time. The paper presents an end-to-end hybrid solution for detecting violence in real-time video frames incorporating both human and weapon detection algorithms applied in a synchronized way. The focus of this article is to propose a generic HWVd (Human Weapon Violence detection) model to detect all kinds of public violence. HWVd is a three-tier ensemble model to detect violence in videos. The first tier is human detection, which uses a LightTrack framework. In the second tier, a Fast Region-based Convolutional Neural Network (F-RCNN) to detect any weapon in videos is used. The third tier uses a pre-trained VGG 19 (a pre-trained model of CNN) for spatial feature extraction and Long Short Term Memory (LSTM) to detect violent activity. Lastly, the output of this framework is sent to the Support Vector Machine to classify the activity as (i) violence not involving weapon, (ii) violence involving weapon and (iii) non-violent. The accuracy obtained using the proposed model is 98%.