Human-robot Teams Research Articles

One Size Does Not Fit All: Mechanisms of Employees’ Acceptance of Robotic Lower-Level Managers

Organizations are facing fundamental changes in digitized work environments. These changes include the introduction of android and humanoid social robots designed to assist employees. The current research addresses whether and under what conditions employees accept such robots as lower-level managers, not just as assistants. The authors extend research on social robots with the expectation-disconfirmation paradigm. They examine how the interplay of expected and experienced performance-related and relational characteristics of robot-supervisors affect employees’ willingness to work with the robotic manager. Results from an online experiment with 7061 U.S. office workers reveal different effects for perceived performance-related and relational characteristics of a robot manager. For performance-related characteristics, employees’ willingness to work with robot managers follows an S-shaped pattern. A slight overfulfillment of expectations is associated with the highest levels of readiness. For relational characteristics, the results instead follow a degressive curve. Increasing positive experiences are associated with decreasing positive evaluations of willingness. Overall, employees indicate a preference for android robots over humanoid robots as managers. This research thus suggests tactics for the successful implementation of mixed human-robot teams and the effective introduction of social robots into lower-level management roles.

Human-Robot Team Performance Compared to Full Robot Autonomy in 16 Real-World Search and Rescue Missions: Adaptation of the DARPA Subterranean Challenge

Human operators in human-robot teams are commonly perceived to be critical for mission success. To explore the direct and perceived impact of operator input on task success and team performance, 16 real-world missions (10 h) were conducted based on the DARPA Subterranean Challenge. Missions involved deploying a heterogeneous team of robots to locate and identify artefacts such as climbing rope, drills and a mannequin representing a human survivor. Two conditions were evaluated: human operators that could control the robot team with state-of-the-art autonomy (Human-Robot Team) compared to autonomous missions without human operator input (Robot-Autonomy). Human interventions included creating waypoints to prioritise high-yield areas, and to navigate through error-prone spaces. Human-Robot Teams were often in directed autonomy mode (70% of mission time), found more items ( \(+\) 10.52%), traversed more distance ( \(+\) 12.71%), covered more unique ground ( \(+\) 10.56%), and longer time between safety-related events (34%). In routine conditions, both condition scores were comparable for artefacts, distance and coverage. Human-Robot Teams were faster at finding the first artefact but slower to respond to information from the robot team. Overall, operators contribute to mission-based outcomes, help to overcome environmental situations that can impede progress, and can assist robots to recover faster from difficult events.

Towards human capability estimation to enhance human-robot team performance

Abstract Skilled labor shortage is a prominent challenge in the world of work. Meanwhile, age-related disabilities or injury lead to at least temporary performance limitations, which make people unfit to work. Consequently, even less workers are available. By employing human-robot teams, the performance of these people may be restored. This requires a good artificial understanding of the human’s capabilities, as generic robot behavior is not feasible with the highly individualized manifestations of disability. We present an approach that allows the robot to autonomously assess human capabilities based on standards from occupational medicine. The method does not only indicate the presence/absence of capabilities, but gives them a discrete rating. This allows the robot to better define its own behavior as a mixture of supportive actions based on gaps in the detailed capabilities.

Human prior knowledge estimation from movement cues for information-based control of mobile robots during search

Robotic search often involves teleoperating vehicles into unknown environments. In such scenarios, prior knowledge of target location or environmental map may be a viable resource to tap into and control other autonomous robots in the vicinity towards an improved search performance. In this paper, we test the hypothesis that despite having the same skill, prior knowledge of target or environment affects teleoperator actions, and such knowledge can therefore be inferred through robot movement. To investigate whether prior knowledge can improve human-robot team performance, we next evaluate an adaptive mutual-information blending strategy that admits a time-dependent weighting for steering autonomous robots. Human-subject experiments show that several features including distance travelled by the teleoperated robot, time spent staying still, speed, and turn rate, all depend on the level of prior knowledge and that absence of prior knowledge increased workload. Building on these results, we identified distance travelled and time spent staying still as movement cues that can be used to robustly infer prior knowledge. Simulations where an autonomous robot accompanied a human teleoperated robot revealed that whereas time to find the target was similar across all information-based search strategies, adaptive strategies that acted on movement cues found the target sooner more often than a single human teleoperator compared to non-adaptive strategies. This gain is diluted with number of robots, likely due to the limited size of the search environment. Results from this work set the stage for developing knowledge-aware control algorithms for autonomous robots in collaborative human-robot teams.

The Journey or the Destination: The Impact of Transparency and Goal Attainment on Trust in Human-Robot Teams

As robots gain autonomy, human-robot task delegation can become more goal-oriented; specifying what to do rather than how. This can lead to unexpected robot behaviour. We investigated the effect of transparency and outcome on the perceived trustworthiness of a robot that deviates from the expected manner to reach a delegated goal. Participants (N = 82) engaged in a virtual military mission as a Human-Robot Team using a 2x2 between-subjects design (low vs. high transparency, positive vs. negative outcome). Participants received training on the expected manner to reach the mission’s goal. In the actual mission, the robot deviated from the planned path. We manipulated whether the robot explained its deviation and whether the outcome was better or worse than the original plan. Results showed that transparency contributed to higher and more stable levels of trust, without increasing subjective workload. While the robot’s deviation led to a violation of trust in the low transparency condition, trust remained stable in the high transparency condition, indicating a buffering effect of transparency on trust in case of unexpected behaviour. The impact of outcome on trust was consistent across transparency conditions. Our findings underscore the role of transparency as a tool for fostering human-robot trust.

Robot learning on the job: Human-in-the-loop autonomy and learning during deployment

With the rapid growth of computing powers and recent advances in deep learning, we have witnessed impressive demonstrations of novel robot capabilities in research settings. Nonetheless, these learning systems exhibit brittle generalization and require excessive training data for practical tasks. To harness the capabilities of state-of-the-art robot learning models while embracing their imperfections, we present Sirius, a principled framework for humans and robots to collaborate through a division of work. In this framework, partially autonomous robots are tasked with handling a major portion of decision-making where they work reliably; meanwhile, human operators monitor the process and intervene in challenging situations. Such a human–robot team ensures safe deployments in complex tasks. Further, we introduce a new learning algorithm to improve the policy’s performance on the data collected from the task executions. The core idea is re-weighing training samples with approximated human trust and optimizing the policies with weighted behavioral cloning. We evaluate Sirius in simulation and on real hardware, showing that Sirius consistently outperforms baselines over a collection of contact-rich manipulation tasks, achieving an 8% boost in simulation and 27% on real hardware than the state-of-the-art methods in policy success rate, with twice faster convergence and 85% memory size reduction. Videos and more details are available at https://ut-austin-rpl.github.io/sirius/ .

TIP: A trust inference and propagation model in multi-human multi-robot teams

Trust is a crucial factor for effective human–robot teaming. Existing literature on trust modeling predominantly focuses on dyadic human-autonomy teams where one human agent interacts with one robot. There is little, if not no, research on trust modeling in teams consisting of multiple human and robotic agents. To fill this important research gap, we present the Trust Inference and Propagation (TIP) model to model and estimate human trust in multi-human multi-robot teams. In a multi-human multi-robot team, we postulate that there exist two types of experiences that a human agent has with a robot: direct and indirect experiences. The TIP model presents a novel mathematical framework that explicitly accounts for both types of experiences. To evaluate the model, we conducted a human-subject experiment with 15 pairs of participants (N=30\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$N=30$$\\end{document}). Each pair performed a search and detection task with two drones. Results show that our TIP model successfully captured the underlying trust dynamics and significantly outperformed a baseline model. To the best of our knowledge, the TIP model is the first mathematical framework for computational trust modeling in multi-human multi-robot teams.

Sociotechnical considerations on developing human robot teaming solutions for construction: a case study

This research advocates for a paradigm shift in the exploration of human–robot teaming solutions for construction automation, by focusing on an integrated view of sociotechnical systems (STS) that recognize the inter-dependencies among actors at various levels when tracing how innovative ideas about intelligent robotic technologies translate into practice in the construction sector. Through a qualitative case study, the paper examines industry and organizational considerations for developing and adopting robotic technologies, leadership vision, mediation, and change management to propose integrative strategies to enhance expectations, acceptance, and deployment of intelligent technologies in human–robot teams (HRTs). This study contributes to research in construction robotics at three organizational levels—macro, meso, and micro. The Integrated Human–Robot Teaming Framework and associated workplan schema offer guidance for navigating human–robot teaming complexities. The study recommends adopting STS principles in planning and deploying robotics applications for construction, emphasizing the integration of multiple elements across the lifecycle. Active leadership and mediation emerge as critical elements in navigating complex networks, ensuring successful outcomes in the dynamic construction environment.

The paradoxical leader, crafting human worker, and robot teammate: A commentary on the future of leader behaviors

Robot utilization is expected to result in significant changes in the way organizations and teams operate. The increasing prevalence of robots in the workplace offers an opportunity to research robot utilization and employee response behaviors to this phenomenon. A timely research question is, “What reactionary behaviors would organizational leaders and employees exhibit as they begin to regard robots as coworkers and teammates?”. The purpose of this commentary is to offer theoretical perspectives on leader response behaviors when leading human-robot teams as well as to understand employee reactions to working alongside a robot teammate. Drawing on paradox and Job Demands-Resources (JD-R) theories and through postulation statements, we show the (contradictory) duality of leader and employee behaviors under conditions of high/low job demands in a human-robot team environment. In doing so, this discussion bears implications for research on team leadership, job crafting, the presence of behavioral robotics, and for the practice of organizational leadership.

Effect of human emotional responses on human-robot team pty in construction

Human-robot collaboration (HRC) is expected to improve construction productivity by integrating robotic capabilities with human expertise. However, the effects of HRC on human emotions are unclear despite their critical impacts on human performance during HRC. This paper examined the inter-relationship between robots' parameters, like movement speed, and emotional responses, during HRC for a construction task. Eighteen participants were observed as they experience various conditions of robots' parameters during a bricklaying task with an arm-type collaborative robot in a laboratory setting. Their brainwaves (i.e., electroencephalogram (EEG) signals) were monitored using an EEG headset to identify emotional responses. Results showed significant emotional variations due to robots' parameters, which could affect the estimated productivity of human-robot teams. This paper demonstrated the emotional changes caused by the robots' parameters during HRC for construction tasks, highlighting the importance of considering emotions as a way to balance the emotional well-being of people and the high productivity of human-robot teams.

Unlocking underrepresented use-cases for large language model-driven human-robot task planning

Large language models (LLM) are now the de facto task planners for Embodied AI (EAI) systems. This shift can be attributed to LLMs' powerful, emergent properties which enable their adaptation to downstream tasks with minimal to no fine tuning via prompting. However, we find that LLM-driven task planning is not a solved problem. In this work we measure the extent to which these models can be adapted to complex and domain-specific task planning via few-shot prompting. Additionally, we contribute quantitative and qualitative analysis on prompt robustness. Lastly, to meet the challenges of adapting EAI systems to real-world, industrial domains, we adopt a human-in-the-loop approach to guarantee safe and interpretable task planning and execution. We successfully demonstrate co-located, human-robot teaming where an Augmented Reality (AR) headset mediates information exchanged between an EAI agent and human operator for a variety of inspection tasks. To our knowledge the use of an AR headset for multimodal grounding and the application of EAI to industrial tasks are novel contributions within Embodied AI research.

Generating Pattern-Based Conventions for Predictable Planning in Human-Robot Collaboration

For humans to effectively work with robots, they must be able to predict the actions and behaviors of their robot teammates rather than merely react to them. While there are existing techniques enabling robots to adapt to human behavior, there is a demonstrated need for methods that explicitly improve humans’ ability to understand and predict robot behavior at multi-task timescales. In this work, we propose a method leveraging the innate human propensity for pattern recognition in order to improve team dynamics in human-robot teams and to make robots more predictable to the humans that work with them. Patterns are a cognitive tool that humans use and rely on often, and the human brain is in many ways primed for pattern recognition and usage. We propose Pattern-Aware Convention-setting for Teaming (PACT), an entropy-based algorithm that identifies and imposes appropriate patterns over a robot’s planner or policy over long time horizons. These patterns are autonomously generated and chosen via an algorithmic process that considers human-perceptible features and characteristics derived from the tasks to be completed, and as such, produces behavior that is easier for humans to identify and predict. Our evaluation shows that PACT contributes to significant improvements in team dynamics and teammate perceptions of the robot, as compared to robots that utilize traditionally ‘optimal’ plans and robots utilizing unoptimized patterns.

HOTSPOT: An ad hoc teamwork platform for mixed human-robot teams

HOTSPOT: An ad hoc teamwork platform for mixed human-robot teams

Enhancing human–robot collaborative transportation through obstacle-aware vibrotactile warning and virtual fixtures

Transporting large and heavy objects can benefit from Human–Robot Collaboration (HRC), increasing the contribution of robots to our daily tasks and addressing challenges arising from labor shortages. This strategy typically positions the human collaborator as the leader, with the robot assuming the follower role. However, when transporting large objects, the operator’s situational awareness can be compromised as the objects may occlude different parts of the environment, weakening the human leader’s decision-making capacity and leading to failure due to collision. This paper proposes a situational awareness framework for collaborative transportation to face this challenge. The framework integrates a multi-modal haptic-based Obstacle Feedback Module with two units. The first unit consists of a warning module that alerts the operator through a haptic belt with four vibrotactile devices that provide feedback about the location and proximity of the obstacles. The second unit implements virtual fixtures as hard constraints for mobility. The warning feedback and the virtual fixtures act online based on the information given by two Lidars mounted on a mobile manipulator to detect the obstacles in the surroundings. By enhancing the operator’s awareness of the environment, the proposed module improves the safety of the human–robot team in collaborative transportation scenarios by preventing collisions. Experiments with 16 non-expert subjects in four feedback modalities during four scenarios report an objective evaluation thanks to quantitative metrics and subjective evaluations based on user-level experiences. The results reveal the strengths and weaknesses of the implemented feedback modalities while providing solid evidence of the increased situational awareness of the operator when the two haptic units are employed.

Optimizing Human-Robot Teaming Performance through Q-Learning-Based Task Load Adjustment and Physiological Data Analysis.

The transition to Industry 4.0 and 5.0 underscores the need for integrating humans into manufacturing processes, shifting the focus towards customization and personalization rather than traditional mass production. However, human performance during task execution may vary. To ensure high human-robot teaming (HRT) performance, it is crucial to predict performance without negatively affecting task execution. Therefore, to predict performance indirectly, significant factors affecting human performance, such as engagement and task load (i.e., amount of cognitive, physical, and/or sensory resources required to perform a particular task), must be considered. Hence, we propose a framework to predict and maximize the HRT performance. For the prediction of task performance during the development phase, our methodology employs features extracted from physiological data as inputs. The labels for these predictions-categorized as accurate performance or inaccurate performance due to high/low task load-are meticulously crafted using a combination of the NASA TLX questionnaire, records of human performance in quality control tasks, and the application of Q-Learning to derive task-specific weights for the task load indices. This structured approach enables the deployment of our model to exclusively rely on physiological data for predicting performance, thereby achieving an accuracy rate of 95.45% in forecasting HRT performance. To maintain optimized HRT performance, this study further introduces a method of dynamically adjusting the robot's speed in the case of low performance. This strategic adjustment is designed to effectively balance the task load, thereby enhancing the efficiency of human-robot collaboration.

Risk-based performance assessment from fully manual to human-robot teaming in pressurized tank inspection operations

The use of robotics in the process industry has shown a stable, increasing trend, according to survey reports, reducing the exposure of operators to critical operations, mainly during maintenance. Moving from full manual (FM) to human-robot teaming (HRT) operations is expected to reduce the risks for operators and increase operational efficiency. The methods used to assess system performances require an adaptation. Traditional risk assessment techniques are no longer adapted to analyze the new way of working; the previous methods proposed for HRT systems have been criticized for their fragmentation, complexity, and lack of validation.This study proposed an integrated framework, including the qualitative and quantitative stages, to investigate the risk-based system performance and compare the full manual and human-robot teaming scenarios in pressurized tank inspection operations. The outputs demonstrated that the integrated framework worked for FM and HRT system performance assessment, considering multiple element types and their interdependencies, generating knowledge that can be exploited to reduce the system's risk and choose among different operational alternatives.

Exploring a GPT-based large language model for variable autonomy in a VR-based human-robot teaming simulation.

In a rapidly evolving digital landscape autonomous tools and robots are becoming commonplace. Recognizing the significance of this development, this paper explores the integration of Large Language Models (LLMs) like Generative pre-trained transformer (GPT) into human-robot teaming environments to facilitate variable autonomy through the means of verbal human-robot communication. In this paper, we introduce a novel simulation framework for such a GPT-powered multi-robot testbed environment, based on a Unity Virtual Reality (VR) setting. This system allows users to interact with simulated robot agents through natural language, each powered by individual GPT cores. By means of OpenAI's function calling, we bridge the gap between unstructured natural language input and structured robot actions. A user study with 12 participants explores the effectiveness of GPT-4 and, more importantly, user strategies when being given the opportunity to converse in natural language within a simulated multi-robot environment. Our findings suggest that users may have preconceived expectations on how to converse with robots and seldom try to explore the actual language and cognitive capabilities of their simulated robot collaborators. Still, those users who did explore were able to benefit from a much more natural flow of communication and human-like back-and-forth. We provide a set of lessons learned for future research and technical implementations of similar systems.

An Evaluation Framework of Human-Robot Teaming for Navigation Among Movable Obstacles via Virtual Reality-Based Interactions

An Evaluation Framework of Human-Robot Teaming for Navigation Among Movable Obstacles via Virtual Reality-Based Interactions

Autonomous Policy Explanations for Effective Human-Machine Teaming

Policy explanation, a process for describing the behavior of an autonomous system, plays a crucial role in effectively conveying an agent's decision-making rationale to human collaborators and is essential for safe real-world deployments. It becomes even more critical in effective human-robot teaming, where good communication allows teams to adapt and improvise successfully during uncertain situations by enabling value alignment within the teams. This thesis proposal focuses on improving human-machine teaming by developing novel human-centered explainable AI (xAI) techniques that empower autonomous agents to communicate their capabilities and limitations via multiple modalities, teach and influence human teammates' behavior as decision-support systems, and effectively build and manage trust in HRI systems.

Who’s in charge here? A survey on Trustworthy AI in Variable Autonomy Robotic Systems

This paper surveys the Variable Autonomy (VA) robotics literature that considers two contributory elements to Trustworthy AI: transparency and explainability. These elements should play a crucial role when designing and adopting robotic systems, especially in VA where poor or untimely adjustments of the system’s level of autonomy can lead to errors, control conflicts, user frustration and ultimate disuse of the system. Despite this need, transparency and explainability is, to the best of our knowledge, mostly overlooked in VA robotics literature or is not considered explicitly. In this paper, we aim to present and examine the most recent contributions to the VA literature concerning transparency and explainability. In addition, we propose a way of thinking about VA by breaking these two concepts down based on: the mission of the human-robot team; who the stakeholder is; what needs to be made transparent or explained; why they need it; and how it can be achieved. Last, we provide insights and propose ways to move VA research forward. Our goal with this paper is to raise awareness and inter-community discussions among the Trustworthy AI and the VA robotics communities.