Human-robot Team Performance Research Articles

Human-Robot Team Performance Compared to Full Robot Autonomy in 16 Real-World Search and Rescue Missions: Adaptation of the DARPA Subterranean Challenge

Human-Robot Team Performance Compared to Full Robot Autonomy in 16 Real-World Search and Rescue Missions: Adaptation of the DARPA Subterranean Challenge

Does the introduction of a cobot change the productivity and posture of the operators in a collaborative task?

Musculoskeletal disorders (MSDs) are the main occupational diseases and are pathologies of multifactorial origin, with posture being one of them. This creates new human-robot collaboration situations that can modify operator behaviors and performance in their task. These changes raise questions about human-robot team performance and operator health. This study aims to understand the consequences of introducing a cobot on work performance, operator posture, and the quality of interactions. It also aims to evaluate the impact of two levels of difficulty in a dual task on these measures. For this purpose, thirty-four participants performed an assembly task in collaboration with a co-worker, either a human or a cobot with two articulated arms. In addition to this motor task, the participants had to perform an auditory task with two levels of difficulty (dual task). They were equipped with seventeen motion capture sensors. The collaborative work was filmed with a camera, and the actions of the participants and co-worker were coded based on the dichotomy of idle and activity. Interactions were coded based on time out, cooperation, and collaboration. The results showed that performance (number of products manufactured) was lower when the participant collaborated with a cobot rather than a human, with also less collaboration and activity time. However, RULA scores were lower-indicating a reduced risk of musculoskeletal disorders-during collaboration with a cobot compared to a human. Despite a decrease in production and a loss of fluidity, likely due to the characteristics of the cobot, working in collaboration with a cobot makes the task safer in terms of the risk of musculoskeletal disorders.

The effect of cognitive workload on decision authority assignment in human–robot collaboration

Human–robot collaboration in dynamic industrial environments warrants robot flexibility and shifting between tasks. Adaptive robot behavior unavoidably carries decision-making needs regarding task allocation and scheduling. Such decisions can be made either by the human team members or autonomously, by the robot’s controlling algorithm. Human authority may help preserve situational awareness but increases mental demands due to increased responsibilities. Conversely, granting authority to the robot can offload the operator, at the cost of reduced intervention readiness. This paper aims to investigate the question of decision authority assignment in a human–robot team, in terms of performance, perceived workload and subjective preference. We hypothesized that the answer is influenced by the cognitive workload imposed on the human operator by the work process. An experiment with 21 participants was conducted, in which decision authority and induced workload through a secondary task were varied between trials. Results confirmed that operators can support the robot better when decision authority is allocated according to their workload. However, operator decision authority (a) may cause inferior performance at any secondary tasks performed in parallel with robot supervision and (b) increases perceived workload. Subjective preference was found to be evenly divided between the two levels of decision authority, and unaffected by task difficulty. In brief, if human–robot team performance is a priority, humans should be granted decision authority when their overall workload allows it. In high-workload conditions, system decision-making algorithms should be developed. Nonetheless, process designers should be mindful of the interpersonal differences between operators who are destined to collaborate with robots.

Reverse Psychology in Trust-Aware Human-Robot Interaction

To facilitate effective human-robot interaction (HRI), trust-aware HRI has been proposed, wherein the robotic agent explicitly considers the human's trust during its planning and decision making. The success of trust-aware HRI depends on the specification of a trust dynamics model and a trust-behavior model. In this study, we proposed one novel trust-behavior model, namely the reverse psychology model, and compared it against the commonly used disuse model. We examined how the two models affect the robot's optimal policy and the human-robot team performance. Results indicate that the robot will deliberately “manipulate” the human's trust under the reverse psychology model. To correct this “manipulative” behavior, we proposed a trust-seeking reward function that facilitates trust establishment without significantly sacrificing the team performance.

Trust-Aware Decision Making for Human-Robot Collaboration

Trust in autonomy is essential for effective human-robot collaboration and user adoption of autonomous systems such as robot assistants. This article introduces a computational model that integrates trust into robot decision making. Specifically, we learn from data a partially observable Markov decision process (POMDP) with human trust as a latent variable. The trust-POMDP model provides a principled approach for the robot to (i) infer the trust of a human teammate through interaction, (ii) reason about the effect of its own actions on human trust, and (iii) choose actions that maximize team performance over the long term. We validated the model through human subject experiments on a table clearing task in simulation (201 participants) and with a real robot (20 participants). In our studies, the robot builds human trust by manipulating low-risk objects first. Interestingly, the robot sometimes fails intentionally to modulate human trust and achieve the best team performance. These results show that the trust-POMDP calibrates trust to improve human-robot team performance over the long term. Further, they highlight that maximizing trust alone does not always lead to the best performance.

Group Interactions with Robots in a Robotics Tournament

In a dynamic environment that involves multiple humans and multiple robots, competitive and collaborative work, and high time pressure, understanding how team players interact with each other and with teleoperated robots has a significant potential to improve human–robot team performance. This study used a naturalistic observation method to explore group interactions with robots during a qualifying robotics tournament. Teams of high school students had six weeks to design, build, and program a robot that plays a field game against other robots. In each match, three robot teams played collaboratively as an alliance against the other three-team alliance, gaining points by defending their stronghold and conquering the opponent’s stronghold. The study identified five areas where groups interacted with robots in a typical tournament (e.g., pit, queuing hall, and match field) and major team roles (e.g., mechanical, programming, electrical work, marketing, and driving). Critical group interactions with robots were captured in several situations, varying according to activity areas and individuals’ roles on their robotic teams. Potential cognitive and emotional issues, future research directions, and implications of the study were discussed.

Human-Robot Mutual Adaptation in Shared Autonomy.

Shared autonomy integrates user input with robot autonomy in order to control a robot and help the user to complete a task. Our work aims to improve the performance of such a human-robot team: the robot tries to guide the human towards an effective strategy, sometimes against the human's own preference, while still retaining his trust. We achieve this through a principled human-robot mutual adaptation formalism. We integrate a bounded-memory adaptation model of the human into a partially observable stochastic decision model, which enables the robot to adapt to an adaptable human. When the human is adaptable, the robot guides the human towards a good strategy, maybe unknown to the human in advance. When the human is stubborn and not adaptable, the robot complies with the human's preference in order to retain their trust. In the shared autonomy setting, unlike many other common human-robot collaboration settings, only the robot actions can change the physical state of the world, and the human and robot goals are not fully observable. We address these challenges and show in a human subject experiment that the proposed mutual adaptation formalism improves human-robot team performance, while retaining a high level of user trust in the robot, compared to the common approach of having the robot strictly following participants' preference.

The impact of human–robot multimodal communication on mental workload, usability preference, and expectations of robot behavior

Multimodal communication between humans and autonomous robots is essential to enhance effectiveness of human–robot team performance in complex, novel environments, such as in military intelligence, surveillance, and reconnaissance operations in urban settings. It is imperative that a systematic approach be taken to evaluate the factors that each modality contributes to the user’s ability to perform successfully and safely. This paper addresses the effects of unidirectional speech and gesture methods of communication on perceived workload, usability preferences, and expectations of robot behavior while commanding a robot teammate to perform a spatial-navigation task. Each type of communication was performed alone or simultaneously. Results reveal that although the speech-alone condition elicited the lowest level of perceived workload, the usability preference and expectations of robot behavior after interacting through each communication condition was the same. Further, workload ratings between the gesture and speech-gesture conditions were similar indicating systems that employ gesture communication could also support speech communication with little to no additional subjectively perceived cognitive burden on the user. Findings also reveal that workload alone should not be used as a sole determining factor of communication preference during system and task evaluation and design. Additionally, perceived workload did not seem to negatively impact the level of expectations regarding the robot’s behavior. Recommendations for future human–robot communication evaluation are provided.

Improved human–robot team performance through cross-training, an approach inspired by human team training practices

We design and evaluate a method of human–robot cross-training, a validated and widely used strategy for the effective training of human teams. Cross-training is an interactive planning method in which team members iteratively switch roles with one another to learn a shared plan for the performance of a collaborative task. We first present a computational formulation of the robot mental model, which encodes the sequence of robot actions necessary for task completion and the expectations of the robot for preferred human actions, and show that the robot model is quantitatively comparable to the mental model that captures the inter-role knowledge held by the human. Additionally, we propose a quantitative measure of robot mental model convergence and an objective metric of model similarity. Based on this encoding, we formulate a human–robot cross-training method and evaluate its efficacy through experiments involving human subjects . We compare human–robot cross-training to standard reinforcement learning techniques, and show that cross-training yields statistically significant improvements in quantitative team performance measures, as well as significant differences in perceived robot performance and human trust. Finally, we discuss the objective measure of robot mental model convergence as a method to dynamically assess human errors. This study supports the hypothesis that the effective and fluent teaming of a human and a robot may best be achieved by modeling known, effective human teamwork practices.

Adaptive Aiding of Human-Robot Teaming

In many emerging civilian and military operations, human operators are increasingly being tasked to supervise multiple robotic uninhabited vehicles (UVs) with the support of automation. As 100% automation reliability cannot be assured, it is important to understand the effects of automation imperfection on performance. In addition, adaptive aiding may help counter any adverse effects of static (fixed) automation. Using a high-fidelity multi-UV simulation involving both air and ground vehicles, two experiments examined the effects of automation reliability and adaptive automation on human-system performance with different levels of task load. In Experiment 1, participants performed a reconnaissance mission while assisted with an automatic target recognition (ATR) system whose reliability was low, medium, or high. Overall human-robot team performance was higher than with either human or ATR performance alone. In Experiment 2, participants performed a similar reconnaissance mission with no ATR, static automation, or with adaptive automation keyed to task load. Participant trust and self-confidence were higher and workload was lower for adaptive automation compared with the other conditions. The results show that human-robot teams can benefit from imperfect static automation even in high task load conditions and that adaptive automation can provide additional benefits in trust and workload.

Editors’ Introduction to the Special Issue on Improving Human-Robot Interaction in Complex Operational Environments

The past decade has seen a proliferation in the use of robots in a broad range of complex domains, such as urban search and rescue, military operations (including explosive ordnance disposal and intelligence, surveillance, and reconnaissance), scientific exploration in underwater and space expeditions, law enforcement (e.g., bomb squads), manufacturing, and health care (e.g., telerobotic surgery). Remotely controlled ground, aerial, sea-surface, and underwater robotic vehicles are serving as tools to safely extend the sensory and psychomotor capabilities of humans to remote environments while keeping humans safe. More importantly, with the ever-increasing technological sophistication in their design and capabilities, robots are becoming more than mere tools. They now can be seen as quasi–team members whose tasks and behaviors must be integrated with the task requirements and expectations of their human teammates. Yet without the necessary research base to inform design, sizeable investments will likely be made into developing robotic systems that are ineffective or, at best, not optimal. Toward this end, the goal of this two-part special issue is to present examples of the latest research on human-robot interaction that demonstrate how theories and empirical findings can be translated into practical, useful guidance for improving human-robot team performance across various domains.

A Research Approach to Shared Mental Models and Situation Assessment in Future Robot Teams

Understanding team cognition requires a multifaceted research approach, aimed at understanding individual and team cognition within and across humans and agents. Measurement and prediction of mental models and situation awareness (SA) of humans, robotic agents, and teams are needed to understand the knowledge required for advanced human-robot team performance. In this paper, we present a research approach to understanding shared mental models (SMMs) and SA with respect to human-robot teams. Having shared and accurate mental models will allow for human and robotic teammates to understand each other’s roles and tasks and also provide a level of predictive behavior that allows teammates to anticipate each other’s needs. Incorporating robotic agents’ input into team SA will improve the adaptive capability of the team. Combined with SA, SMMs can provide a deeper understanding of the happenings in the mission environment.

Experimenting False Alarm Demand for Human Robot Interactions in Humanoid Soccer Robots

Performance of robots in human robot teams has always been a topic of interest for many researchers in human robot interaction community. Traditionally adopted neglect tolerance model for performance measurements assume ideal conditions in which the operator switches control between robots sequentially based on acceptable performance level for each robot ignoring any false alarms due to erroneous interactions. In this paper, we present the false alarm demand, a new metric for measuring effects of false alarms on human robot team performance and extend the neglect tolerance model to situations in which false positives and false negatives are prevalent. Experiments were performed with real and virtual humanoid soccer robots across tele-operated, and point to point modes of autonomy. Measured false alarm demand and robot performances were largely consistent with the proposed extended neglect tolerance model predictions for real and virtual experiments for both autonomy modes. Experiments also showed that extended neglect tolerance model offered better estimation of robot performances as compared to the traditionally adopted neglect tolerance model for situations wherein false alarms are prevalent.

Evaluating Situation Awareness in Human-Robot Teams

New methodologies and quantitative measurements for evaluating human-robot team performance must be developed to achieve effective coordination between teams of humans and unmanned vehicles. The Mixed Initiative Team Performance Assessment System (MITPAS) provides such a comprehensive measurement methodology. MITPAS consists of a methodology, tools and procedures to measure the performance of mixed manned and unmanned teams in both training and real world operational environments. This paper shows results of an initial experiment conducted to validate the Situation Awareness Global Assessment Technique (SAGAT) methodology as part of the MITPAS tool and gain insight into the effect of robot competence on operator situation awareness as well as on overall human-robot team performance.

A Comprehensive Methodology for Assessing Human-Robot Team Performance for Use in Training and Simulation

New methodologies and quantitative measurements for evaluating human-robot team performance must be developed to achieve effective coordination between teams of humans and unmanned vehicles. The Mixed Initiative Team Performance Assessment System (MITPAS) provides such a comprehensive measurement methodology. MITPAS consists of a methodology, tools and procedures to measure the performance of mixed manned and unmanned teams in both training and real world operational environments. This paper describes MITPAS and the results of an initial experiment conducted to validate the measures and gain insight into the effect of robot competence on operator trust as well as on human-robot team performance.