HDRLM3D: A Deep Reinforcement Learning-Based Model with Human-like Perceptron and Policy for Crowd Evacuation in 3D Environments

Abstract

At present, a common drawback of crowd simulation models is that they are mainly simulated in (abstract) 2D environments, which limits the simulation of crowd behaviors observed in real 3D environments. Therefore, we propose a deep reinforcement learning-based model with human-like perceptron and policy for crowd evacuation in 3D environments (HDRLM3D). In HDRLM3D, we propose a vision-like ray perceptron (VLRP) and combine it with a redesigned global (or local) perceptron (GOLP) to form a human-like perception model. We propose a double-branch feature extraction and decision network (DBFED-Net) as the policy, which can extract features and make behavioral decisions. Moreover, we validate our method’s ability to reproduce typical phenomena and behaviors through experiments in two different scenarios. In scenario I, we reproduce the bottleneck effect of crowds and verify the effectiveness and advantages of HDRLM3D by comparing it with real crowd experiments and classical methods in terms of density maps, fundamental diagrams, and evacuation times. In scenario II, we reproduce agents’ navigation and obstacle avoidance behaviors and demonstrate the advantages of HDRLM3D for crowd simulation in unknown 3D environments by comparing it with other deep reinforcement learning-based models in terms of trajectories and numbers of collisions.