Spatiotemporal feature based convolutional neural network for violence detection

Abstract

Automatic violence detection is one of the most interesting tasks in video surveillance applications. It is essential to ensure human safety by preventing fatal accidents. In this paper, we propose a spatiotemporal feature based on convolutional neural network and optical flow information for violence detection. Our descriptor, named distribution-based CNN feature (DCNN), estimates first the joint distribution of the optical flow magnitude and orientation around STIP points in the aim of modeling the motion structure. Then, it extracts deep features using the pretrained ResNet50 network. Our DCNN feature is finally fed into the SVM classifier for training. We evaluate the proposed descriptor on two challenging benchmark datasets designed for violence detection in both uncrowded and crowded scenes. For the two datasets, our DCNN feature shows significant improvement compared to the state-of-the-art descriptors.