A Deep Learning Approach to Predict Crowd Behavior Based on Emotion

Abstract

In a visual surveillance system, predicting crowd behavior has recently emerged as a crucial problem for crowd management and monitoring. Specifically, potential dangers and disasters can be avoided by correctly detecting crowd behavior. In this paper, we propose an approach to forecast crowd behavior using a deep learning framework and multiclass Support Vector Machine (SVM). We extract spatio-temporal descriptors using 3D Convolutional Neural Network (3DCNN) based on crowd emotions. In particular, the learned emotion based descriptors help to build the semantic ambiguity in classifying crowd behavior. The effectiveness of our approach is validated with 3 benchmark datasets: Motion Emotion Dataset (MED), ViolentFlows and UMN. The obtained results prove that our approach is successful in predicting crowd behavior in challenging situations. Our system also outperforms existing methods that use local feature descriptors, which reveals that emotions from spatio-temporal features are beneficial for the correct anticipation of crowd behavior.