Crowd Navigation Research Articles

Research on deep reinforcement learning-based complex crowd navigation algorithm using risk perception and path selection

Abstract Aiming at the Frozen Robot Problem and the poor effect of dynamic obstacle avoidance in traditional navigation methods when encountering dynamic obstacles, a deep reinforcement learning navigation method based on risk perception and path selection is proposed. The core of this method lies in CP (Collision Probability) module and IOU (Intersection over Union) module. The CP module calculates the CP between the robot and the nearby dynamic obstacles in real time, so that the robot can avoid the dangerous obstacles first. At the same time, the intersection-merging ratio module calculates the “pass ability” of the area near the robot in real time, and guides the robot to choose a safer area to pass through. Compared with the DRL method without these two modules, the proposed method achieves the highest navigation success rate in all simulation test environments, and the maximum improvement rate is 11%. At the same time, it still has advantages in navigation time and other indicators.

EmoiPlanner: Human emotion and intention aware socially acceptable robot navigation in human‐centric environments

AbstractThe deployment of robots in human‐centric environments has significantly increased in recent years. It is crucial for robots to navigate human environments while understanding social norms and personal boundaries to ensure a harmonious coexistence between humans and robots. A socially aware robot should be capable of interpreting and responding appropriately to human cues, expressions, and intentions, thereby fostering trust and confidence among humans. However, prior studies were insufficient or unable to address the navigation challenges in human‐populated environments, as they perceive people as obstacles rather than social agents. Recent studies have utilized proxemic areas that are present in interpersonal interactions for human‐robot interaction scenarios, but they have enforced consistent proxemic areas for social robot navigation. This approach fails to fully capture the highly sophisticated behaviour and preferences of humans. Therefore, we propose a psychologically‐based adaptive proxemic area that fluctuates based on the human's emotional state. Furthermore, we integrate this feature into a reinforcement learning‐based social navigation framework, making our navigation framework robust to the unpredictable affections of humans. Additionally, our navigation framework includes human intention prediction to approximate the future proxemic area, thereby avoiding interference with the path to be taken by individuals. We have named our framework the Human Emotion and Intention Aware Path Planner (EmoiPlanner). Our framework has been subjected to case studies involving realistic crowd navigation scenarios, and the results indicate that it enables robots to navigate through crowds without causing discomfort to pedestrians who exhibit stochastic behaviours and emotional states, while also ensuring efficient path planning.

Crowd navigation for dynamic hazard avoidance in evacuation using emotional reciprocal velocity obstacles

Crowd evacuation can be a challenging task, especially in emergency situations involving dynamically moving hazards. Effective obstacle avoidance is crucial for successful crowd evacuation, particularly in scenarios involving dynamic hazards such as natural or man-made disasters. In this paper, we propose a novel application of the emotional reciprocal velocity obstacles (ERVO) method for obstacle avoidance in dynamic hazard scenarios. ERVO is an established method that incorporates agent emotions and obstacle avoidance to produce more efficient and effective crowd navigation. Our approach improves on previous research by using ERVO to model the perceptive danger posed by dynamic hazards in real-time, which is crucial for rapid response in emergency situations. We conducted experiments to evaluate our approach and compared our results with other velocity obstacle methods. Our findings demonstrate that our approach is able to improve agent coordination, reduce congestion, and produce superior avoidance behavior. Our study shows that incorporating emotional reciprocity into obstacle avoidance can enhance crowd behavior in dynamic hazard scenarios.

Safe and socially compliant robot navigation in crowds with fast-moving pedestrians via deep reinforcement learning

AbstractSafe and socially compliant navigation in a crowded environment is essential for social robots. Numerous research efforts have shown the advantages of deep reinforcement learning techniques in training efficient policies, while most of them ignore fast-moving pedestrians in the crowd. In this paper, we present a novel design of safety measure, named Risk-Area, considering collision theory and motion characteristics of different robots and humans. The geometry of Risk-Area is formed based on the real-time relative positions and velocities of the agents in the environment. Our approach perceives risk in the environment and encourages the robot to take safe and socially compliant navigation behaviors. The proposed method is verified with three existing well-known deep reinforcement learning models in densely populated environments. Experiment results demonstrate that our approach combined with the reinforcement learning techniques can efficiently perceive risk in the environment and navigate the robot with high safety in the crowds with fast-moving pedestrians.

Automatic Crowd Navigation Path Planning in Public Scenes Through Multiobjective Differential Evolution

Crowd navigation path planning is important in public scenes. Existing strategies are mainly based on manual design, which is not flexible or effective enough. This article proposes an evolutionary framework for automatic crowd navigation path planning in public scenes. The proposed framework contains a new fitness evaluation mechanism that can quantitatively evaluate the quality of a path planning strategy by considering both crowd safety and flow speed. Based on the fitness evaluation mechanism, a framework based on multiobjective differential evolution (DE) is developed to efficiently evolve path planning strategies. Simulation results on two synthetic scenes and a real-world metro station scene show that the proposed framework can provide good path planning strategies.

Uncertainty-Aware DRL for Autonomous Vehicle Crowd Navigation in Shared Space

Uncertainty-Aware DRL for Autonomous Vehicle Crowd Navigation in Shared Space

Modular hierarchical reinforcement learning for multi-destination navigation in hybrid crowds

Real-world robot applications usually require navigating agents to face multiple destinations. Besides, the real-world crowded environments usually contain dynamic and static crowds that implicitly interact with each other during navigation. To address this challenging task, a novel modular hierarchical reinforcement learning (MHRL) method is developed in this paper. MHRL is composed of three modules, i.e., destination evaluation, policy switch, and motion network, which are designed exactly according to the three phases of solving the original navigation problem. First, the destination evaluation module rates all destinations and selects the one with the lowest cost. Subsequently, the policy switch module decides which motion network to be used according to the selected destination and the obstacle state. Finally, the selected motion network outputs the robot action. Owing to the complementary strengths of a variety of motion networks and the cooperation of modules in each layer, MHRL is able to deal with hybrid crowds effectively. Extensive simulation experiments demonstrate that MHRL achieves better performance than state-of-the-art methods.

STP4: spatio temporal path planning based on pedestrian trajectory prediction in dense crowds

This article proposes a means of autonomous mobile robot navigation in dense crowds based on predicting pedestrians’ future trajectories. The method includes a pedestrian trajectory prediction for a running mobile robot and spatiotemporal path planning for when the path crosses with pedestrians. The predicted trajectories are converted into a time series of cost maps, and the robot achieves smooth navigation without dodging to the right or left in crowds; the path planner does not require a long-term prediction. The results of an evaluation implementing this method in a real robot in a science museum show that the trajectory prediction works. Moreover, the proposed planning’s arrival times is 26.4% faster than conventional 2D path planning’s arrival time in a simulation of navigation in a crowd of 50 people.

SafeCrowdNav: safety evaluation of robot crowd navigation in complex scenes.

Navigating safely and efficiently in dense crowds remains a challenging problem for mobile robots. The interaction mechanisms involved in collision avoidance require robots to exhibit active and foresighted behaviors while understanding the crowd dynamics. Deep reinforcement learning methods have shown superior performance compared to model-based approaches. However, existing methods lack an intuitive and quantitative safety evaluation for agents, and they may potentially trap agents in local optima during training, hindering their ability to learn optimal strategies. In addition, sparse reward problems further compound these limitations. To address these challenges, we propose SafeCrowdNav, a comprehensive crowd navigation algorithm that emphasizes obstacle avoidance in complex environments. Our approach incorporates a safety evaluation function to quantitatively assess the current safety score and an intrinsic exploration reward to balance exploration and exploitation based on scene constraints. By combining prioritized experience replay and hindsight experience replay techniques, our model effectively learns the optimal navigation policy in crowded environments. Experimental outcomes reveal that our approach enables robots to improve crowd comprehension during navigation, resulting in reduced collision probabilities and shorter navigation times compared to state-of-the-art algorithms. Our code is available at https://github.com/Janet-xujing-1216/SafeCrowdNav.

Graph Relational Reinforcement Learning for Mobile Robot Navigation in Large-Scale Crowded Environments

Mobile robot autonomous navigation in large-scale environments with crowded dynamic objects and static obstacles is still an essential yet challenging task. Recent works have demonstrated the potential of using deep reinforcement learning to enable autonomous navigation in crowds. However, only considering the human-robot interactions results in short-sighted and unsafe behaviors, and they typically use hand-crafted features and assume the global observation range, leading to large performance declines in large-scale crowded environments. Recent advances have shown the power of graph neural networks to learn local interactions among surrounding objects. In this paper, we consider autonomous navigation task in large-scale environments with crowded static and dynamic objects (such as humans). Particularly, local interactions among dynamic objects are learned for better-understanding their moving tendency and relational graph learning is introduced for aggregating both the object-object interactions and object-robot interactions. In addition, local observations are transformed into graphical inputs to achieve the scalability to various number of surrounding dynamic objects and various static obstacle patterns, and the globally guided reinforcement learning strategy is introduced to achieve the fixed-sized learning model even in large-scale complex environments. Simulation results validate our generalizability to various environments and advanced performance compared with existing works in large-scale crowded environments. In particular, our method with only local observations performs better than the benchmarks with global complete observability. Finally, physical robotic experiments demonstrate our effectiveness and practical applicability in real scenarios.

Social crowd navigation of a mobile robot based on human trajectory prediction and hybrid sensing

Social crowd navigation of a mobile robot based on human trajectory prediction and hybrid sensing

ViewBirdiformer: Learning to Recover Ground-Plane Crowd Trajectories and Ego-Motion From a Single Ego-Centric View

We introduce a novel learning-based method for view birdification [1], the task of recovering ground-plane trajectories of pedestrians of a crowd and their observer in the same crowd just from the observed ego-centric video. View birdification becomes essential for mobile robot navigation and localization in dense crowds where the static background is hard to see and reliably track. It is challenging mainly for two reasons; i) absolute trajectories of pedestrians are entangled with the movement of the observer which needs to be decoupled from their observed relative movements in the ego-centric video, and ii) a crowd motion model describing the pedestrian movement interactions is specific to the scene yet unknown a priori. For this, we introduce a Transformer-based network referred to as ViewBirdiformer which implicitly models the crowd motion through self-attention and decomposes relative 2D movement observations onto the ground-plane trajectories of the crowd and the camera through cross-attention between views. Most important, ViewBirdiformer achieves view birdification in a single forward pass which opens the door to accurate real-time, always-on situational awareness. Extensive experimental results demonstrate that ViewBirdiformer achieves accuracy similar to or better than state-of-the-art with three orders of magnitude reduction in execution time.

Navigating Robots in Dynamic Environment With Deep Reinforcement Learning

In the fight against COVID-19, many robots replace human employees in various tasks that involve a risk of infection. Among these tasks, the fundamental problem of navigating robots among crowds, named robot crowd navigation, remains open and challenging. Therefore, we propose HGAT-DRL, a heterogeneous GAT-based deep reinforcement learning algorithm. This algorithm encodes the constrained human-robot-coexisting environment in a heterogeneous graph consisting of four types of nodes. It also constructs an interactive agent-level representation for objects surrounding the robot, and incorporates the kinodynamic constraints from the non-holonomic motion model into the deep reinforcement learning (DRL) framework. Simulation results show that our proposed algorithm achieves a success rate of 92%, at least 6% higher than four baseline algorithms. Furthermore, the hardware experiment on a Fetch robot demonstrates our algorithm’s successful and convenient migration to real robots.

Robot navigation in crowds via deep reinforcement learning with modeling of obstacle uni-action

Mobile robots operating in public environments require the ability to navigate among humans and obstacles in a socially compliant and safe manner. Previous work has shown the power of deep reinforcement learning (DRL) techniques by employing them to train efficient policies for robot navigation. However, most DRL-based robot navigation methods only consider dynamic pedestrians and do not take static obstacles into account. The ability to approach pedestrians and obstacles differently will improve a robot's navigation efficiency. In this work, we propose a novel network, the obstacle-robot uni-action (ORU) network, to encode the one-way direct effects of obstacles on the robot. Obstacles' indirect effects on the robot, represented by the obstacle-human uni-action (OHU), together with human–human interaction (HHI), are concatenated to a human–robot interaction (HRI) network to obtain the features of a crowd's effects on the robot. We also implement a variable reaching goal reward and an approaching goal reward in our model, which can enhance the method's performance in terms of navigation time. Results of test experiments in both simulation and real datasets demonstrate that the proposed method outperforms state-of-the-art methods.

Robots' Motion Planning in Human Crowds by Acceleration Obstacles

We study the Acceleration Obstacle (AO) as a concept to enable a robot's navigation in human crowds. The AO's geometric properties are analyzed and a direct sampling-free algorithm is proposed to approximate its boundary by linear constraints. The resulting controller is formulated as a quadratic program and evaluated in interaction with simulated bi-directional crowd flow in a corridor. We compare it to alternative robotic controllers, considering the robot's and the crowd's performance and the robot's behavior with respect to emergent lanes. Our results indicate that the robot can achieve higher efficiency when being less integrated in lanes.

A Unified Multiple-Motion-Mode Framework for Socially Compliant Navigation in Dense Crowds

Mobile robots are expected to move safely and efficiently in socially-compliant ways in dense crowds. In this paper, a unified multiple-motion-mode framework is proposed to achieve socially-compliant navigation in dense crowds. The proposed framework consists of three successive phases, the identification of pedestrian groups and prediction of their trajectories, the generation of candidate local trajectories of the robot, and the determination of an optimal local trajectory. The pedestrians are grouped according to distances among pedestrians and their velocities and a predictor predicts the groups’ trajectories. Three robot motion modes, moving-solo, pedestrian-following, and courteous-stopping, are proposed based on a systematic analysis and classification of dense crowds and they are utilized to generate candidate local trajectories. A composite metric is proposed to determine the best local trajectory by simultaneously considering the robot motion safety, efficiency, and stability of the trajectory. The proposed robot navigation framework is evaluated both in simulations and real-world experiments with various scenarios. The results show that the proposed framework surpasses the state-of-the-art methods in terms of efficiency and social compliance. Note to Practitioners—This paper is motivated by the socially-compliant navigation problem of mobile robots in densely crowded environments such as hospitals and airports. Robots navigation in these scenes need to follow collective social conventions to ensure ordered and efficient public pedestrian traffics, and individual social conventions for acceptance by human society. Existing approaches generally consider only one type of social conventions cannot accomplish a truly socially-compliant navigation and cannot be naturally accepted in the human pedestrian environment. This paper proposes to imitate the pedestrian’s walking behavior for robot navigation by naturally choosing one of the three walking modes, moving-solo, following, and courteous-stopping. We construct the candidate trajectory sampling strategies for different motion modes based on the global path and the two types of social conventions. Experimental results show that the proposed framework can be applied in large-scale, densely crowded, unstructured, and human-robot coexisting environments for socially-compliant navigation. However, the proposed navigation framework has limited improvement in navigation efficiency in a highly crowded environment due to the limited feasible space. Even pedestrians are difficult to walk in these highly crowded environments. In future work, we will study how to auto-adjust the weights to steer the robot to efficient navigation in highly crowded environments.

Eye contact avoidance in crowds: A large wearable eye-tracking study

Eye contact is essential for human interactions. We investigated whether humans are able to avoid eye contact while navigating crowds. At a science festival, we fitted 62 participants with a wearable eye tracker and instructed them to walk a route. Half of the participants were further instructed to avoid eye contact. We report that humans can flexibly allocate their gaze while navigating crowds and avoid eye contact primarily by orienting their head and eyes towards the floor. We discuss implications for crowd navigation and gaze behavior. In addition, we address a number of issues encountered in such field studies with regard to data quality, control of the environment, and participant adherence to instructions. We stress that methodological innovation and scientific progress are strongly interrelated.

Deep Reinforcement Learning Based on Social Spatial–Temporal Graph Convolution Network for Crowd Navigation

In the crowd navigation, reinforcement learning based on graph neural network is a promising method, which effectively solves the poor navigation effect based on social interaction model and the freezing behavior of robot in extreme cases. However, since the information correlation of human trajectory has not been involved in the method, its performance still needs improvement. Therefore, we proposed a deep reinforcement learning model based on Social Spatial–Temporal Graph Convolution Network (SSTGCN) to handle the crowd navigation problem, in which the spatial–temporal information of human trajectory has been taken advantage to predict human behavior intentions and help robot plan path more efficiently. The model consists of graph learning module and robot forward planning module. In the graph learning module, the latent features of agents are taken advantage to reason about the relations among the agents, and SSTGCN is used to update feature matrix. In addition, value estimation module calculates state representation and state prediction module predicts the next state. The robot forward planning module makes use of k-step planning to estimate the quality of state and searches the best k steps planning. We tested our model in the Crowd-Nav platform, and the results show that our model has high navigation success rate and short navigation time. In addition, it has good robustness to crowd changes.

Crowd navigation in an unknown and complex environment based on deep reinforcement learning

AbstractWe propose a virtual crowd navigation approach based on deep reinforcement learning to improve the adaptability of virtual crowds in an unknown and complex environment. To address the problem of local optimum or slow iteration or even failure to converge due to sparse rewards in complex environments, we integrate the curiosity‐driven mechanism, the key navigation points acquisition and the failure path penalty method in addition to combining long short‐term memory networks, dynamic obstacle collision prediction with proximal policy optimization algorithms, which realizes the crowd navigation in a complex environment. The experimental results show that the proposed approach can simulate the motions of virtual crowds in various dynamic and complex scenarios without the environment modeling. The use of continuous action space also ensures that the movement trajectories of the virtual crowds are more realistic and natural. Furthermore, our approach can provide analysis and demonstration tools for a variety of independent collaborations such as competition and cooperation of group intelligence in an open and dynamic environment.

Robot navigation in a crowd by integrating deep reinforcement learning and online planning

Navigating mobile robots along time-efficient and collision-free paths in crowds is still an open and challenging problem. The key is to build a profound understanding of the crowd for mobile robots, which is the basis of a proactive and foresighted policy. However, since the interaction mechanisms among pedestrians are complex and sophisticated, it is difficult to describe and model them accurately. For the excellent approximation capability of deep neural networks, deep reinforcement learning is a promising solution to the problem. However, current model-free learning-based approaches in crowd navigation always neglect planning and still lead to reactive collision avoidance policies and shortsighted behaviors. Meanwhile, most model-based learning-based approaches are based on state values, imposing a substantial computational burden. To address these problems, we propose a graph-based deep reinforcement learning method, social graph-based double dueling deep Q-network (SG-D3QN), that (i) introduces a social attention mechanism to extract an efficient graph representation for the crowd-robot state, (ii) extends the previous state value approximator to a state-action value approximator, (iii) further optimizes the state-action value approximator with simulated experiences generated by the learned environment model, and (iv) then proposes a human-like decision-making process by integrating model-free reinforcement learning and online planning. Experimental results indicate that our approach helps the robot understand the crowd and achieves a high success rate of more than 0.99 in the crowd navigation task. Compared with previous state-of-the-art algorithms, our approach achieves better performance. Furthermore, with the human-like decision-making process, our approach incurs less than half of the computational cost.