Human-aware Navigation Research Articles

Bridging Requirements, Planning, and Evaluation: A Review of Social Robot Navigation.

Navigation lies at the core of social robotics, enabling robots to navigate and interact seamlessly in human environments. The primary focus of human-aware robot navigation is minimizing discomfort among surrounding humans. Our review explores user studies, examining factors that cause human discomfort, to perform the grounding of social robot navigation requirements and to form a taxonomy of elementary necessities that should be implemented by comprehensive algorithms. This survey also discusses human-aware navigation from an algorithmic perspective, reviewing the perception and motion planning methods integral to social navigation. Additionally, the review investigates different types of studies and tools facilitating the evaluation of social robot navigation approaches, namely datasets, simulators, and benchmarks. Our survey also identifies the main challenges of human-aware navigation, highlighting the essential future work perspectives. This work stands out from other review papers, as it not only investigates the variety of methods for implementing human awareness in robot control systems but also classifies the approaches according to the grounded requirements regarded in their objectives.

Fuzzy inference system-assisted human-aware navigation framework based on enhanced potential field

With the advent of Autonomous Mobile Robots (AMRs) in public areas such as malls and airports, their harmonious coexistence with humans is crucial. AMRs must operate in a manner that ensures human safety, comfort, and acceptability to reduce stress. This is called Human Aware Navigation. This study introduces a framework for AMR navigation that prioritizes safety and human comfort in such environments, utilizing an enhanced Potential Field approach augmented by Fuzzy Inference Systems. To achieve a smooth AMR trajectory, the framework employs these systems based on AMR, human, and obstacle information. The proposed approach is tested across various scenarios, including complex, cluttered environments that mimic practical situations. Simulation results demonstrate that AMRs using the proposed method navigate human-rich environments safely and comfortably while mitigating common issues associated with Potential Field-based approaches, such as local minima and obstacles near the goal.

A novel Human-Aware Navigation algorithm based on behavioral intention cognition

AbstractIn order to ensure safe and comfortable human–robot navigation in close proximity, it is imperative for robots to possess the capability to understand human behavioral intention. With this objective in mind, this paper introduces a Human-Aware Navigation (HAN) algorithm. The HAN system combines insights from studies on human detection, social behavioral model, and behavior prediction, all while incorporating social distance considerations. This information is integrated into a layer dedicated to human behavior intention cognition, achieved through the fusion of data from laser radar and Kinect sensors, employing Gaussian functions to account for individual private space and movement trend. To cater to the mapping requirements of the HAN system, we have reduced the computational complexity associated with traditional multilayer cost map by implementing a “first-come, first-served” expansion method. Subsequently, we have enhanced the trajectory optimization equation by incorporating an improved dynamic triangle window method that integrates human behavior intention cognition, leading to the determination of an appropriate trajectory for the robot. Finally, experimental evaluations have been conducted to assess and validate the efficacy of the human behavior intention cognition and the HAN system. The results clearly demonstrate that the HAN system outperforms the traditional Dynamic Window Approach algorithm in ensuring the safety and comfort of humans in human–robot coexistence environments.

Implementation of a human‐aware robot navigation module for cooperative soft‐fruit harvesting operations

AbstractIn the last decades, robotic solutions have been introduced in agriculture to improve the efficiency of tasks such as spraying, plowing, and seeding. However, for a more complex task like soft‐fruit harvesting, the efficiency of experienced human pickers has not been surpassed yet by robotic solutions. Thus, in the immediate future, human labor will probably be still necessary for picking tasks while robotic platforms could be used as collaborators, supporting the pickers in the transportation of the harvested fruit. This cooperative harvesting strategy creates a human–robot interaction (HRI) that requires significant further development in human‐aware safe navigation and effective bidirectional communication of intent. In fact, although agricultural robots are considered small/medium size machinery, they still represent a risk of causing injuries to human collaborators, especially if people are not trained to work with robots or robot operations are not intuitive. Avoiding such injury is the aim of this work which contributes to the development, implementation, and evaluation of a human‐aware navigation (HAN) module that can be integrated into the autonomous navigation system of commercial agricultural robots. The proposed module is responsible for the detection and monitoring of humans working around the robot and uses this information to activate safety actions depending on whether the human presence is considered at risk or not. Apart from ensuring a physically safe HRI, the proposed module deals with the comfort level and psychological safety of human coworkers. The latter is possible by using an explicit human–robot communication strategy that lets both know of the other's intentions, increasing the level of trust and reducing inefficient pauses triggered by unnecessary safety actions. The proposed HAN solution was integrated into a commercial agricultural robot and tested in several situations that are expected to happen during cooperative harvesting operations. The results of an usability assessment illustrated the benefits of the proposal in terms of safety, efficiency, and ergonomics.

Toward Benchmarking of Long-Term Spatio-Temporal Maps of Pedestrian Flows for Human-Aware Navigation.

Despite the advances in mobile robotics, the introduction of autonomous robots in human-populated environments is rather slow. One of the fundamental reasons is the acceptance of robots by people directly affected by a robot’s presence. Understanding human behavior and dynamics is essential for planning when and how robots should traverse busy environments without disrupting people’s natural motion and causing irritation. Research has exploited various techniques to build spatio-temporal representations of people’s presence and flows and compared their applicability to plan optimal paths in the future. Many comparisons of how dynamic map-building techniques show how one method compares on a dataset versus another, but without consistent datasets and high-quality comparison metrics, it is difficult to assess how these various methods compare as a whole and in specific tasks. This article proposes a methodology for creating high-quality criteria with interpretable results for comparing long-term spatio-temporal representations for human-aware path planning and human-aware navigation scheduling. Two criteria derived from the methodology are then applied to compare the representations built by the techniques found in the literature. The approaches are compared on a real-world, long-term dataset, and the conception is validated in a field experiment on a robotic platform deployed in a human-populated environment. Our results indicate that continuous spatio-temporal methods independently modeling spatial and temporal phenomena outperformed other modeling approaches. Our results provide a baseline for future work to compare a wide range of methods employed for long-term navigation and provide researchers with an understanding of how these various methods compare in various scenarios.

Impact of decision-making system in social navigation

Facing human activity-aware navigation with a cognitive architecture raises several difficulties integrating the components and orchestrating behaviors and skills to perform social tasks. In a real-world scenario, the navigation system should not only consider individuals like obstacles. It is necessary to offer particular and dynamic people representation to enhance the HRI experience. The robot’s behaviors must be modified by humans, directly or indirectly. In this paper, we integrate our human representation framework in a cognitive architecture to allow that people who interact with the robot could modify its behavior, not only with the interaction but also with their culture or the social context. The human representation framework represents and distributes the proxemic zones’ information in a standard way, through a cost map. We have evaluated the influence of the decision-making system in human-aware navigation and how a local planner may be decisive in this navigation. The material developed during this research can be found in a public repository (https://github.com/IntelligentRoboticsLabs/social_navigation2_WAF) and instructions to facilitate the reproducibility of the results.

Multigoal Visual Navigation With Collision Avoidance via Deep Reinforcement Learning

Learning to map the images acquired by a moving agent equipped with a camera sensor to motion commands for multigoal navigation is challenging. Most existing approaches are still struggling against collision avoidance, faster convergence, and generalization. In this article, a novel actor–critic architecture is presented to learn the optimal navigation policy. We introduce single-step reward observation and collision penalty to reshape the reinforcement learning (RL) reward function. The collision perception can be obtained by the reshaped reward function and treated as measurement information from the visual observation to avoid obstacles. Besides, expert trajectories are used to generate subgoals. A subgoal reward shaping is then proposed to accelerate policy learning with the expert knowledge of subgoals. In order to generate human-aware navigation policies, an observation-action consistency (OAC) model is introduced to ensure that the agent reaches the subgoals in turn, and moves toward the target. The whole training process is performed on a self-supervised RL approach, accompanied by an expert supervision signal. This method balances the exploration and exploitation, helping the proposed model to generalize to unseen goals. The training experiments on AI2-THOR show better performance and faster convergence speed, compared with the existing approaches. For the generalization capacity to unseen goals, the proposed method achieves the state-of-the-art success rate, with at least a 30% improvement of average episode collision.

Dynamic Waypoint Navigation: Model-Based Adaptive Trajectory Planner for Human-Symbiotic Mobile Robots

Path planning in dynamic environments is still a challenging issue with autonomous mobile robots. Current methods lack adaptability to various passing scenarios, a variety of passing trajectories including an acceleration path, or immediacy in planning time, which require human-aware navigation. In this study, we propose Dynamic Waypoint Navigation (DWN), which is a model-based adaptive real-time trajectory planning method. DWN first predicts human-robot path interference and the time and position of the interference on the basis of the measured velocity of humans. It then dynamically designates several waypoints considering the time delay of both calculation time and robot travel time. Then, DWN generates several trajectories by combining different speeds (default, acceleration, and deceleration) and paths (default, right, and left) and selects the best trajectory in terms of an interference-avoidance energy cost based on the degree of velocity-vector change. DWN can also output a trajectory within 0.5 s to immediately adapt to changes in human behavior and adopt a simple mathematical model and algorithm to enable easy expansion. Simulation and experimental results reveal that the DWN can adequately select a time-efficient trajectory in real-time and adaptively change a trajectory depending on human movement.

Evaluating the use of human aware navigation in industrial robot arms

Abstract Although the principles followed by modern standards for interaction between humans and robots follow the First Law of Robotics popularized in science fiction in the 1960s, the current standards regulating the interaction between humans and robots emphasize the importance of physical safety. However, they are less developed in another key dimension: psychological safety. As sales of industrial robots have been increasing over recent years, so has the frequency of human–robot interaction (HRI). The present article looks at the current safety guidelines for HRI in an industrial setting and assesses their suitability. This article then presents a means to improve current standards utilizing lessons learned from studies into human aware navigation (HAN), which has seen increasing use in mobile robotics. This article highlights limitations in current research, where the relationships established in mobile robotics have not been carried over to industrial robot arms. To understand this, it is necessary to focus less on how a robot arm avoids humans and more on how humans react when a robot is within the same space. Currently, the safety guidelines are behind the technological advance, however, with further studies aimed at understanding HRI and applying it to newly developed path finding and obstacle avoidance methods, science fiction can become science fact.

A new human-aware robot navigation framework based on time-dependent social interaction spaces: An application to assistive robots in caregiving centers

One of the most critical problems to be addressed by future generation socially-assistive robots working in semi-organized social environments, such as shopping centers, nursing homes, airports, hospitals, or assisted living centers, is the capability of human-aware navigation. Autonomous navigation in a complex environment with people, staff with different roles, timetables, and restrictions to access, among others, requires adapting to socially accepted rules. Consequently, the path-planner must consider concepts related to proxemics and personal spaces of interaction that include human–human, human–robot, or human–object combinations. Likewise, the speed of approaching people, both to initiate communication or to navigate nearby, must be adapted to social conventions. Some of these situations have already been studied in the literature with varying degrees of success. However, the concept of time dependency or chronemics in the robot social navigation has been poorly explored. Current algorithms do not take into account the social complexity of real environments and their relationship with the time of day or the activities performed in these scenarios. This article presents a new framework for robot social navigation in human environments, introducing the concept of time-dependent social mapping. The main novelty is that the social route planned by the robot considers variables that depend on the time and the scheduled center activities. The article describes how the areas of interaction vary over time and how they affect human-aware navigation. To this end, the proposed navigation stack defines a new function for time-dependent social interaction space that takes continuous values and is configurable by the center’s staff. The global path-planner uses this function to choose dynamically a socially accepted path to the target. Then, the framework uses an elastic band path optimizer as a local planner, adapting the robot’s navigation to possible changes during the trajectory. Several use cases in simulated caregiving centers have been explored to validate the robot’s social navigation improvements using these temporal variables.

Determining Shape and Size of Personal Space of a Human when Passed by a Robot

Autonomous mobile robots that operate in environments with people are expected to be able to deal with human proxemics and social distances. Previous research investigated how robots can approach persons or how to implement human-aware navigation algorithms. However, experimental research on how robots can avoid a person in a comfortable way is largely missing. The aim of the current work is to experimentally determine the shape and size of personal space of a human passed by a robot. In two studies, both a humanoid as well as a non-humanoid robot were used to pass a person at different sides and distances, after which they were asked to rate their perceived comfort. As expected, perceived comfort increases with distance. However, the shape was not circular: passing at the back of a person is more uncomfortable compared to passing at the front, especially in the case of the humanoid robot. These results give us more insight into the shape and size of personal space in human–robot interaction. Furthermore, they can serve as necessary input to human-aware navigation algorithms for autonomous mobile robots in which human comfort is traded off with efficiency goals.

A graph neural network to model disruption in human-aware robot navigation

Autonomous navigation is a key skill for assistive and service robots. To be successful, robots have to minimise the disruption caused to humans while moving. This implies predicting how people will move and complying with social conventions. Avoiding disrupting personal spaces, people’s paths and interactions are examples of these social conventions. This paper leverages Graph Neural Networks to model robot disruption considering the movement of the humans and the robot so that the model built can be used by path planning algorithms. Along with the model, this paper presents an evolution of the dataset SocNav1 (Manso et al 2020) which considers the movement of the robot and the humans, and an updated scenario-to-graph transformation which is tested using different Graph Neural Network blocks. The model trained achieves close-to-human performance in the dataset. In addition to its accuracy, the main advantage of the approach is its scalability in terms of the number of social factors that can be considered in comparison with handcrafted models. The dataset and the model are available in a public repository (https://github.com/gnns4hri/sngnnv2).

Human Aware Navigation System Based on Human Action Recognition

Human Aware Navigation System Based on Human Action Recognition

Multiple human trajectory prediction and cooperative navigation modeling in crowded scenes

As mobile robots start operating in environments crowded with humans, human-aware navigation is required to make these robots navigate safely, efficiently and in socially compliant manner. People navigate in an interactive and cooperative fashion so that, they are able to find their path to a destination even if there is no clear route leading to it. There are significant efforts to solve this problem for mobile robots; however, they are not scalable to high human density and learning based approaches depend heavily on the context and configuration of the set they are trained with. We develop a method which infers initial trajectories from Gaussian processes and updates these trajectories jointly for all agents using a cost based interaction approach. We condition Gaussian processes online with the best hypothesis at each step of prediction horizon. The method is tested on a common public dataset and it is shown that it outperforms two state-of-the-art approaches in terms of human-likeness of predicted trajectories.

A human-aware navigation method for social robot based on multi-layer cost map

Most of the current human-aware navigation methods of service robots focus on improving the reactive navigation of local path planning without considering the global environment. A global path planning method is proposed based on the global scope of pedestrian perception and multi-layer cost-maps. Firstly, personal space and group interaction are modeled as social cost based on pedestrian perception, and then multi-layer dynamic cost maps are generated containing the social cost at different time-steps based on pedestrian trajectory prediction, which provides social constraints for global path planning. Secondly, the global path planner searches for the optimal state with heuristic cost function based on the multi-layer dynamic cost-maps. Considering the huge calculation of heuristic search and the limitation of the length of trajectory prediction duration, the ‘plan-prediction-execution’ cycle is introduced for the dynamic planning, which improves performance in the dynamic environment. Finally, compared with the traditional path planner in the simulation scenes including pedestrian movements and group interaction, the experimental results show that the path length, the execution time is shorter, and the comfort distance of the person/group is more social in our method. Through the actual scene experiments, the advantages of handling situations of planning timeout and adjusting trajectories dynamically after introducing the ‘plan-prediction-execution’ cycle are verified, which can meet the comfort and society of human-aware navigation.

SocNav1: A Dataset to Benchmark and Learn Social Navigation Conventions

Datasets are essential to the development and evaluation of machine learning and artificial intelligence algorithms. As new tasks are addressed, new datasets are required. Training algorithms for human-aware navigation is an example of this need. Different factors make designing and gathering data for human-aware navigation datasets challenging. Firstly, the problem itself is subjective, different dataset contributors will very frequently disagree to some extent on their labels. Secondly, the number of variables to consider is undetermined culture-dependent. This paper presents SocNav1, a dataset for social navigation conventions. SocNav1 aims at evaluating the robots’ ability to assess the level of discomfort that their presence might generate among humans. The 9280 samples in SocNav1 seem to be enough for machine learning purposes given the relatively small size of the data structures describing the scenarios. Furthermore, SocNav1 is particularly well-suited to be used to benchmark non-Euclidean machine learning algorithms such as graph neural networks. This paper describes the proposed dataset and the method employed to gather the data. To provide a further understanding of the nature of the dataset, an analysis and validation of the collected data are also presented.

A Preliminary Study of Interactive Navigation Framework with Situation-Adaptive Multimodal Inducement: Pass-By Scenario

Human-aware navigation is an essential requirement for autonomous robots in human-coexisting environments. The goal of conventional navigation is to find a path for a robot to pass through safely and efficiently without colliding with human. Note that if such a path cannot be found, the robot stops until a path is clear. Thus, such collision-avoidance based passive navigation does not work in a congested or narrow space. To avoid this freezing problem, the robot should induce humans to make a space for passing by an adequate inducement method, such as body movement, speech, and touch, depending on the situation. A robot that deliberately clears a path with such actions may make humans uncomfortable, so the robot should also utilize inducements to avoid causing negative feelings. In this study, we propose a fundamental framework of interactive navigation with situation-adaptive multimodal inducement. For a preliminary study, we target a passing scenario in a narrow corridor where two humans are standing and adopt a model-based approach focusing on common parameters. The suitable inducement basically varies depending on the largest space through which a robot can pass, distance between the robot and a human, and human behavior such as conversing. We thus develop a situation-adaptive inducement selector on the basis of the relationship between human–robot proximity and allowable inducement strength, considering robot efficiency and human psychology. The proposed interactive navigation system was tested across some contextual scenarios and compared with a fundamental path planner. The experimental results indicated that the proposed system solved freezing problems, provided a safe and efficient trajectory, and improved humans’ psychological reaction although the evidence was limited to robot planner and hardware design we used as well as certain scenes, contexts, and participants.

Efficient and robust Pedestrian Detection using Deep Learning for Human-Aware Navigation

This paper addresses the problem of Human-Aware Navigation (HAN), using multi camera sensors to implement a vision-based person tracking system. The main contributions of this paper are as follows: a novel and efficient Deep Learning person detection and a standardization of human-aware constraints. In the first stage of the approach, we propose to cascade the Aggregate Channel Features (ACF) detector with a deep Convolutional Neural Network (CNN) to achieve fast and accurate Pedestrian Detection (PD). Regarding the human awareness (that can be defined as constraints associated with the robot’s motion), we use a mixture of asymmetric Gaussian functions, to define the cost functions associated to each constraint. Both methods proposed herein are evaluated individually to measure the impact of each of the components. The final solution (including both the proposed pedestrian detection and the human-aware constraints) is tested in a typical domestic indoor scenario, in four distinct experiments. The results show that the robot is able to cope with human-aware constraints, defined after common proxemics and social rules.

Socially aware robot navigation system in human-populated and interactive environments based on an adaptive spatial density function and space affordances

Traditionally robots are mostly known by society due to the wide use of manipulators, which are generally placed in controlled environments such as factories. However, with the advances in the area of mobile robotics, they are increasingly inserted into social contexts, i.e., in the presence of people. The adoption of socially acceptable behaviours demands a trade-off between social comfort and other metrics of efficiency. For navigation tasks, for example, humans must be differentiated from other ordinary objects in the scene. In this work, we propose a novel human-aware navigation strategy built upon the use of an adaptive spatial density function that efficiently cluster groups of people according to their spatial arrangement. Space affordances are also used for defining potential activity spaces considering the objects in the scene. The proposed function defines regions where navigation is either discouraged or forbidden. To implement a socially acceptable navigation, the navigation architecture combines a probabilistic roadmap and rapidly-exploring random tree path planners, and an adaptation of the elastic band algorithm. Trials in real and simulated environments carried out demonstrate that the use of the clustering algorithm and social rules in the navigation architecture do not hinder the navigation performance.

Multi robot collision avoidance in a shared workspace

This paper presents a decentralised human-aware navigation algorithm for shared human–robot work-spaces based on the velocity obstacles paradigm. By extending our previous work on collision avoidance, we are able to include and avoid static and dynamic obstacles, no matter whether they are induced by other robots and humans passing through. Using various cost maps and Monte Carlo sampling with different cost factors accounting for humans and robots, the approach allows human workers to use the same navigation space as robots. It does not rely on any external positioning sensors and shows its feasibility even in densely packed environments.