Abnormal Crowd Behavior Detection Using Motion Information Images and Convolutional Neural Networks

Cem Direkoglu

doi:10.1109/access.2020.2990355

Abstract

We introduce a novel method for abnormal crowd event detection in surveillance videos. Particularly, our work focuses on panic and escape behavior detection that may appear because of violent events and natural disasters. First, optical flow vectors are computed to generate a motion information image (MII) for each frame, and then MIIs are used to train a convolutional neural network (CNN) for abnormal crowd event detection. The proposed MII is a new formulation that provides a visual appearance of crowd motion. The proposed MIIs make the discrimination between normal and abnormal behaviors easier. The MII is mainly based on the optical flow magnitude, and angle difference computed between the optical flow vectors in consecutive frames. A CNN is employed to learn normal and abnormal crowd behaviors using MIIs. The MII generation, and the combination with a CNN is a new approach in the context of abnormal crowd behavior detection. Experiments are performed on commonly used datasets such as UMN and PETS2009. Evaluation indicates that our method achieves the best results.

Highlights

Analysis of crowd behavior has become a popular research field in recent years
The proposed work is compared to the existing works in this domain such as Optical Flow Features (OFF) [14], the method based on Bayesian model (BM) [18], sparse reconstruction cost (SRC) [1], chaotic invariants (CI) [11], the social force model (SF) [7], the force field model (FF) [33], behaviour Entropy model (BE) [13], distribution of magnitude of optical flow (DMOF) [19], context location and motion-rich spatiotemporal volumes (CL and MSV) [20], generative adversarial nets GAN [21], temporal convolutional neural network (CNN) Pattern (TCP) [22], global event influence model (GEIM) [23], and histograms of optical flow orientation and magnitude (HOFO) [24]
We presented an approach for abnormal crowd behaviour detection

Summary

INTRODUCTION

Analysis of crowd behavior has become a popular research field in recent years. Crowd behavior analysis can be utilized in variety of applications, for example, automatic detection of panic and escape behavior as a result of violence, riots, natural disasters, and so forth. Crowd behavior inside a global scene is abnormal, such as sudden escape of people during an earthquake. This work focuses on global abnormal crowd behavior detection. A. RELATED WORK For global crowd behavior analysis usually holistic and object-based methods are utilized. In holistic methods [5]–[7], the crowd is considered as a global unit These approaches analyzes the whole crowd itself to extract useful features (e.g. applying optical flow to the entire frame) in order to detect the crowd behavior. We concentrate on global abnormal crowd event detection in surveillance videos, for example, sudden escape of people in the same or diverse directions.

Direkoglu

MOTION INFORMATION IMAGE GENERATION

EVALUATION AND RESULTS

CONCLUSIONS