Spatial-temporal Consistency based Crowd Emotion Recognition

Abstract

In crowd evacuation scenarios, it is an effective way to ensure evacuation safety by recognizing real crowd emotions and then taking measures such as emotional infection to reduce crowd panic. However, factors such as distance, exposure, angle, and occlusion can lead to incomplete face expression information collection, thus failing to accurately identify crowd emotions. Therefore, it is still a very challenging problem to recognize individual emotions in the case of incomplete face expression information collection, and thus accurately identify crowd emotions during evacuation. To solve this problem, we propose a spatial-temporal consistency-based crowd emotion recognition method to accurately identify the real emotions of the crowd. First, for video frames that can capture the complete facial expression information, we use the residual network to accurately identify the individual emotion values in each frame. For video frames that cannot capture the complete facial expression information, we propose an individual emotion calculation model based on spatial-temporal consistency to calculate the individual emotion values in each frame. Second, we define the crowd panic level and obtain the real crowd emotion by calculation. Finally, we implement an end-to-end crowd panic emotion recognition system to verify our method. The experimental results show that the method can accurately calculate the crowd panic level, which is important for guiding crowd evacuation.