Data-driven crowd evacuation: A reinforcement learning method

Abstract

Traditional methods of crowd evacuation simulation mainly focus on improving crowd evacuation efficiency while many assumptions reduce the visual realism of crowd evacuation. The data-driven method is an effective way to enhance the visual realism of crowd simulation whose main idea is to approach the trajectories of crowd movements and social attributes in real video as much as possible. However, the existing work lacks consideration of variable scenarios and dynamic movements which decreases the visual realism of crowd simulation.To address this problem, we present a reinforcement learning based data-driven crowd evacuation (RL-DCE) framework. In the proposed framework, a data-driven crowd evacuation (DCE) model is established first. The model extracts the dynamic characteristics (such as the position and the velocity) from videos to quantify the cohesiveness which is one of the most important social attributes of crowds. Then, we propose a cohesiveness based K-means (C-K-means) algorithm to group the crowd and merge the individual’s trajectories according to the groups. Second, a hierarchical path planning mechanism is proposed. In this mechanism, the top-layer employs the Q-learning algorithm (which is one of the reinforcement learning algorithms) to train the control policy using the obtained grouping trajectories. The Q-learning algorithm allows individuals to learn the main observed features of crowd motion (e.g., cohesiveness) and demonstrate robustness with respect to dynamic environment. The bottom-layer is mainly in charge of obtaining individual paths and using the reciprocal velocity obstacles (RVO) model to avoid collision. Finally, we implement a crowd simulation system based on the DCE model in dynamic environments. The experimental results show that the proposed method can simulate the crowd evacuation more realistically in the dynamic environment.