Human-robot Interaction Community Research Articles

Error-related potential-based shared autonomy via deep recurrent reinforcement learning

Objective. Error-related potential (ErrP)-based brain–computer interfaces (BCIs) have received a considerable amount of attention in the human–robot interaction community. In contrast to traditional BCI, which requires continuous and explicit commands from an operator, ErrP-based BCI leverages the ErrP, which is evoked when an operator observes unexpected behaviours from the robot counterpart. This paper proposes a novel shared autonomy model for ErrP-based human–robot interaction. Approach. We incorporate ErrP information provided by a BCI as useful observations for an agent and formulate the shared autonomy problem as a partially observable Markov decision process. A recurrent neural network-based actor-critic model is used to address the uncertainty in the ErrP signal. We evaluate the proposed framework in a simulated human-in-the-loop robot navigation task with both simulated users and real users. Main results. The results show that the proposed ErrP-based shared autonomy model enables an autonomous robot to complete navigation tasks more efficiently. In a simulation with 70% ErrP accuracy, agents completed the task 14.1% faster than in the no ErrP condition, while with real users, agents completed the navigation task 14.9% faster. Significance. The evaluation results confirmed that the shared autonomy via deep recurrent reinforcement learning is an effective way to deal with uncertain human feedback in a complex human–robot interaction task.

SocNavBench: A Grounded Simulation Testing Framework for Evaluating Social Navigation

The human-robot interaction community has developed many methods for robots to navigate safely and socially alongside humans. However, experimental procedures to evaluate these works are usually constructed on a per-method basis. Such disparate evaluations make it difficult to compare the performance of such methods across the literature. To bridge this gap, we introduce SocNavBench , a simulation framework for evaluating social navigation algorithms. SocNavBench comprises a simulator with photo-realistic capabilities and curated social navigation scenarios grounded in real-world pedestrian data. We also provide an implementation of a suite of metrics to quantify the performance of navigation algorithms on these scenarios. Altogether, SocNavBench provides a test framework for evaluating disparate social navigation methods in a consistent and interpretable manner. To illustrate its use, we demonstrate testing three existing social navigation methods and a baseline method on SocNavBench , showing how the suite of metrics helps infer their performance trade-offs. Our code is open-source, allowing the addition of new scenarios and metrics by the community to help evolve SocNavBench to reflect advancements in our understanding of social navigation.

Aerial Flight Paths for Communication.

This article presents an understanding of naive users’ perception of the communicative nature of unmanned aerial vehicle (UAV) motions refined through an iterative series of studies. This includes both what people believe the UAV is trying to communicate, and how they expect to respond through physical action or emotional response. Previous work in this area prioritized gestures from participants to the vehicle or augmenting the vehicle with additional communication modalities, rather than communicating without clear definitions of the states attempting to be conveyed. In an attempt to elicit more concrete states and better understand specific motion perception, this work includes multiple iterations of state creation, flight path refinement, and label assignment. The lessons learned in this work will be applicable broadly to those interested in defining flight paths, and within the human-robot interaction community as a whole, as it provides a base for those seeking to communicate using non-anthropomorphic robots. We found that the Negative Attitudes towards Robots Scale (NARS) can be an indicator of how a person is likely to react to a UAV, the emotional content they are likely to perceive from a message being conveyed, and it is an indicator for the personality characteristics they are likely to project upon the UAV. We also see that people commonly associate motions from other non-verbal communication situations onto UAVs. Flight specific recommendations are to use a dynamic retreating motion from a person to encourage following, use a perpendicular motion to their field of view for blocking, simple descending motion for landing, and to use either no motion or large altitude changes to encourage watching. Overall, this research explores the communication from the UAV to the bystander through its motion, to see how people respond physically and emotionally.

Robots beyond Science Fiction: mutual learning in human\u2013robot interaction on the way to participatory approaches

Putting laypeople in an active role as direct expert contributors in the design of service robots becomes more and more prominent in the research fields of human–robot interaction (HRI) and social robotics (SR). Currently, though, HRI is caught in a dilemma of how to create meaningful service robots for human social environments, combining expectations shaped by popular media with technology readiness. We recapitulate traditional stakeholder involvement, including two cases in which new intelligent robots were conceptualized and realized for close interaction with humans. Thereby, we show how the robot narrative (impacted by science fiction, the term robot itself, and assumptions on human-like intelligence) together with aspects of power balancing stakeholders, such as hardware constraints and missing perspectives beyond primary users, and the adaptivity of robots through machine learning that creates unpredictability, pose specific challenges for participatory design processes in HRI. We conclude with thoughts on a way forward for the HRI community in developing a culture of participation that considers humans when conceptualizing, building, and using robots.

Discovery Report Following 5 Years of Research Project on Socially Assistive Robotics

This study contributes to the field of socially assistive robotics (SAR). We focus our research on the perception and projections of SAR and the interconnections between their tasks, roles, appearances, and behaviors. We explore how humanoid and non-humanoid appearance and behavior influence people’s expectations and roles they give to robots. We also tackle broader questions related to projections, identification, and interpretations of SAR in everyday life. Regarding robots’ roles, first studies on the “robotic etiquette” showed that older population saw companion robots as butlers, disabled people see robots as assistants, and only young participants gave roles as being a friend. Even if SAR has as main goal the rehabilitation or medical care, the devices built in this field will also enter people’s homes. Based on appearance and intention scenarios of human-robot experiences, design researchers propose opportunities of projection and discussions and push the human-robot interaction (HRI) community to taking into consideration the more subtle and unspoken needs, values, and expectations on social robots. A larger perspective should be considered in order to deeply understand how to well-design SAR for people and society. This report presents the results of qualitative and quantitative experimentations in a 5-year research project on socially assistive robotics—Romeo consortium. We conducted a mixed study on 72 participants on the perception, projection, and identification of three robot representations (2 humanoid and one non-humanoid). Results show strong correlations between the appearance, behavior, and roles of the robot representations.

Passive Brain-Computer Interfaces for Enhanced Human-Robot Interaction.

Brain-computer interfaces (BCIs) have long been seen as control interfaces that translate changes in brain activity, produced either by means of a volitional modulation or in response to an external stimulation. However, recent trends in the BCI and neurofeedback research highlight passive monitoring of a user's brain activity in order to estimate cognitive load, attention level, perceived errors and emotions. Extraction of such higher order information from brain signals is seen as a gateway for facilitation of interaction between humans and intelligent systems. Particularly in the field of robotics, passive BCIs provide a promising channel for prediction of user's cognitive and affective state for development of a user-adaptive interaction. In this paper, we first illustrate the state of the art in passive BCI technology and then provide examples of BCI employment in human-robot interaction (HRI). We finally discuss the prospects and challenges in integration of passive BCIs in socially demanding HRI settings. This work intends to inform HRI community of the opportunities offered by passive BCI systems for enhancement of human-robot interaction while recognizing potential pitfalls.

Are You Still With Me? Continuous Engagement Assessment From a Robot's Point of View.

Continuously measuring the engagement of users with a robot in a Human-Robot Interaction (HRI) setting paves the way toward in-situ reinforcement learning, improve metrics of interaction quality, and can guide interaction design and behavior optimization. However, engagement is often considered very multi-faceted and difficult to capture in a workable and generic computational model that can serve as an overall measure of engagement. Building upon the intuitive ways humans successfully can assess situation for a degree of engagement when they see it, we propose a novel regression model (utilizing CNN and LSTM networks) enabling robots to compute a single scalar engagement during interactions with humans from standard video streams, obtained from the point of view of an interacting robot. The model is based on a long-term dataset from an autonomous tour guide robot deployed in a public museum, with continuous annotation of a numeric engagement assessment by three independent coders. We show that this model not only can predict engagement very well in our own application domain but show its successful transfer to an entirely different dataset (with different tasks, environment, camera, robot and people). The trained model and the software is available to the HRI community, at https://github.com/LCAS/engagement_detector, as a tool to measure engagement in a variety of settings.

On the impact of different types of errors on trust in human-robot interaction

Abstract Trust is a key dimension of human-robot interaction (HRI), and has often been studied in the HRI community. A common challenge arises from the difficulty of assessing trust levels in ecologically invalid environments: we present in this paper two independent laboratory studies, totalling 160 participants, where we investigate the impact of different types of errors on resulting trust, using both behavioural and subjective measures of trust. While we found a (weak) general effect of errors on reported and observed level of trust, no significant differences between the type of errors were found in either of our studies. We discuss this negative result in light of our experimental protocols, and argue for the community to move towards alternative methodologies to assess trust.

Bridging Multilevel Time Scales in HRI

In this article, we present a multi-level time scales framework for the analysis of human-robot interaction (HRI). Such a framework allows HRI scientists to model the inter-relation between measures and factors of an experiment. Our final goal with the introduction of this framework is to unify scientific practice in the HRI community for better reproducibility. Our new approach transposes Newell’s framework of human actions to model human-robot interaction. Measures from the interaction are sorted into categories (time scales) corresponding to the temporal constraints proposed by Newell. According to this sorting, a bottom-up or top-down analysis can then be performed to correlate variables which allows a better understanding and explanation of the interaction. The utilization of our method within two experimental use cases is then presented. The first one, a child-robot interaction, involves two robots and one child playing a memory game. The second is based on an analysis of the PInSoRo dataset, involving 30 child-robot pairs in a freeplay interaction. Finally, we introduce clear guidelines to re-use the framework.

Escaping Oz: Autonomy in Socially Assistive Robotics

The field of socially assistive robotics (SAR) aims to supplement the efforts of clinicians, therapists, educators, and caregivers through individualized, socially mediated interventions with robots. SAR is faced with the interdisciplinary challenge to balance sensitive domain needs with current technical limitations. Many researchers in SAR and the broader human–robot interaction community overcome technical barriers by using a Wizard of Oz approach, or teleoperation of the robot or aspects of the interaction. Although Wizard of Oz is a well-established practice, it becomes intractable in critical SAR domains that require long-term, situated support, such as aging in place and special needs education. In this article, we define a set of autonomy-centric design properties for SAR interventions based on concepts from artificial intelligence and robotics. These properties structure a systematic review of the last decade of autonomous SAR research. From the review, we draw and discuss common computational methods, engineering practices, and design patterns that enable autonomy in SAR.

Understanding and Resolving Failures in Human-Robot Interaction: Literature Review and Model Development

While substantial effort has been invested in making robots more reliable, experience demonstrates that robots operating in unstructured environments are often challenged by frequent failures. Despite this, robots have not yet reached a level of design that allows effective management of faulty or unexpected behavior by untrained users. To understand why this may be the case, an in-depth literature review was done to explore when people perceive and resolve robot failures, how robots communicate failure, how failures influence people's perceptions and feelings toward robots, and how these effects can be mitigated. Fifty-two studies were identified relating to communicating failures and their causes, the influence of failures on human-robot interaction (HRI), and mitigating failures. Since little research has been done on these topics within the HRI community, insights from the fields of human computer interaction (HCI), human factors engineering, cognitive engineering and experimental psychology are presented and discussed. Based on the literature, we developed a model of information processing for robotic failures (Robot Failure Human Information Processing, RF-HIP), that guides the discussion of our findings. The model describes the way people perceive, process, and act on failures in human robot interaction. The model includes three main parts: (1) communicating failures, (2) perception and comprehension of failures, and (3) solving failures. Each part contains several stages, all influenced by contextual considerations and mitigation strategies. Several gaps in the literature have become evident as a result of this evaluation. More focus has been given to technical failures than interaction failures. Few studies focused on human errors, on communicating failures, or the cognitive, psychological, and social determinants that impact the design of mitigation strategies. By providing the stages of human information processing, RF-HIP can be used as a tool to promote the development of user-centered failure-handling strategies for HRIs.

Reframing Assistive Robots to Promote Successful Aging

We are living in an exciting time, as people are living longer, more active lives. This is reshaping how we think about aging. Rather than viewing aging as a problem to be fixed (i.e., a deficit model of aging), many aging researchers are viewing aging as a developmental stage of life to be celebrated and supported, that is, “successful aging.” In this article, we embrace this approach and consider it in the context of assistive robot design in an aim to steer the conversation away from deficit models that have limited robot design possibilities. To explore an alternative design approach to the study of aging in human-robot interaction (HRI), we invited five aging researchers (three geriatricians, one gerontologist, and one epidemiologist) and nine older adults to participate in our research. In the study, participants illustrated their interpretations of aging and suggested potential assistive robots. We found that while all participants perceived the importance of potential disabilities due to aging, they considered potential disabilities as only one aspect of the experience of aging. They highlighted other key themes to consider in designing robots to support successful aging, such as older adults’ autonomy and resilience. We discuss these findings for the HRI community and call for “robots for successful aging.”

Editorial Introduction: Impact, Sustainability, and Inclusion for JHRI

Dear colleagues in the HRI community, JHRI needs you!It gives us great pleasure to join the Journal of Human-Robot Interaction (JHRI) as the new Editors-in-Chief. We are truly honored to serve the human-robot interaction (HRI) community in this role, and brimming with excitement for the journal in the coming years.

Benefits of Human - Robot Interaction [TC Spotlight

The Technical Committee (TC) on Human - Robot Interaction (HRI) is 12 years old. The next ICRA in St. Paul, Minnesota, United States, on 14?18 May 2012 will be the time for the TC triennial review. We propose that notwithstanding the many results achieved during the past years, the TC in HRI will continue to play an important role in coordinating activities related to HRI, consolidating HRI communities, and pushing research toward new and underexplored areas. We envision several possible directions for future research: 1) new research for developing smart and natural human?robot interfaces 2) increasing the interdisciplinary nature of research in HRI by enlarging the HRI community, especially by linking and coordinating activities with sister groups such as human?computer interaction or human?machine interaction 3) investigating further the social impact of HRI 4) promoting field tests on HRI 5) new knowledge on performance evaluation and benchmarking. We invite all people with an interest in HRI to send us their ideas about future directions, activities, and proposals for renewing this TC.

Human-Robot Interaction

The widespread adoption of personal service robots will likely depend on how well they interact with users. This chapter was motivated by a desire to facilitate the design of usable personal service robots. Toward that end, this chapter reviews the literature concerning people interacting with personal service robots. First, ongoing research related to the design of personal service robots is discussed. This material is organized around generic activities that would take place when a user initiates interaction with a future personal service robot, for example, understanding the robot’s affordances or its cognitive capabilities, as well as when a personal service robot initiates interaction with a user, for example, understanding the user’s intent or engaging and communicating with the user. Second, research areas that deserve more attention from the human-robot interaction community are discussed, for example, understanding when people do and do not treat robots as if they were people. Throughout the chapter, recommendations for the design of future personal service robots are offered along with recommendations for future research.

Validating extended neglect tolerance model for human robot interactions in humanoid soccer robots

SUMMARYEstimating robot performance in human robot teams is a vital problem in human robot interaction community. In a previous work, we presented extended neglect tolerance model for estimation of robot performance, where the human operator switches control between robots sequentially based on acceptable performance levels, taking into account any false alarms in human robot interactions. Task complexity is a key parameter that directly impacts the robot performance as well as the false alarms occurrences. In this paper, we validate the extended neglect tolerance model for two robot tasks of varying complexity levels. We also present the impact of task complexity on robot performance estimations and false alarms demands. Experiments were performed with real and virtual humanoid soccer robots across tele-operated and semi-autonomous modes of autonomy. Measured false alarm demand and robot performances were largely consistent with the extended neglect tolerance model predictions for both real and virtual robot experiments. Experiments also showed that the task complexity is directly proportional to false alarm demands and inversely proportional to robot performance.

False Alarm Metrics for Human–Robot Interactions in Service Robots

Human–robot teams combining the complementary capabilities of robots and humans towards solving potentially complex service tasks are gaining widespread popularity. Many of these tasks will involve close interactions between the robot and the human it serves, thereby making safety a crucial parameter. Erroneous interactions that inevitably arises between the human and the robot cause accidents in service robotic applications. Currently, there are no metrics available in the human–robot interaction community for analyzing erroneous interactions. In this paper, we put forward a new class of false alarm metrics to define, classify and quantify the effects of erroneous interactions in human robot teams, and explore the relationship between false alarms and safety in service robots. We extend the receiver operating characteristics (ROC) curve commonly used in the signal processing community to classify robots based on their associated risks. We also show the utility of the designed false alarm metrics and extended ROC curve by applying them to a service robot, Robo-Erectus@Home, across tele-operation and semi-autonomous modes of autonomy.

Experimenting False Alarm Demand for Human Robot Interactions in Humanoid Soccer Robots

Performance of robots in human robot teams has always been a topic of interest for many researchers in human robot interaction community. Traditionally adopted neglect tolerance model for performance measurements assume ideal conditions in which the operator switches control between robots sequentially based on acceptable performance level for each robot ignoring any false alarms due to erroneous interactions. In this paper, we present the false alarm demand, a new metric for measuring effects of false alarms on human robot team performance and extend the neglect tolerance model to situations in which false positives and false negatives are prevalent. Experiments were performed with real and virtual humanoid soccer robots across tele-operated, and point to point modes of autonomy. Measured false alarm demand and robot performances were largely consistent with the proposed extended neglect tolerance model predictions for real and virtual experiments for both autonomy modes. Experiments also showed that extended neglect tolerance model offered better estimation of robot performances as compared to the traditionally adopted neglect tolerance model for situations wherein false alarms are prevalent.

Cooperative use of unmanned sea surface and micro aerial vehicles at Hurricane Wilma

AbstractOn Oct. 24, 2005, Hurricane Wilma, a category 5 storm, made landfall at Cape Romano, Florida. Three days later, the Center for Robot‐Assisted Search and Rescue at the University of South Florida deployed an iSENYS helicopter and a prototype unmanned water surface vehicle, AEOS‐1, to survey damage in parts of Marco Island, 14 km from landfall. The effort was the first known use of unmanned sea surface vehicles (USVs) for emergency response and established their suitability for the recovery phase of disaster management by detecting damage to seawalls and piers, locating submerged debris (moorings and handrails), and determining safe lanes for sea navigation. It provides a preliminary domain theory of postdisaster port and littoral inspection with unmanned vehicles for use by the human–robot interaction community. It was also the first known demonstration of the strongly heterogeneous USV–micro aerial vehicle (MAV) team for any domain. The effort identified cooperative UAV–USV strategies and open issues for autonomous operations near structures. The effort showed that the MAV provided a much‐needed external view for situation awareness and provided spotting for areas to be inspected. Concepts of operations for USV damage inspection and USV–MAV cooperation emerged, including a formula for computing the human–robot ratio: Nh = (2 × Nv) + 1, where Nh is the number of humans and Nv is the number of vehicles. The outstanding research issues span three areas: challenges for USVs operating near littoral structures, general issues for USV–MAV cooperation, and new applications. It is expected that the lessons learned will be transferrable to defense and homeland safety and security applications, such as port security, and other phases of emergency response, including rescue. © 2008 Wiley Periodicals, Inc.

Final Report for the DARPA/NSF Interdisciplinary Study on Human–Robot Interaction

As part of a Defense Advanced Research Projects Agency/National Science Foundation study on human-robot interaction (HRI), over sixty representatives from academia, government, and industry participated in an interdisciplinary workshop, which allowed roboticists to interact with psychologists, sociologists, cognitive scientists, communication experts and human-computer interaction specialists to discuss common interests in the field of HRI, and to establish a dialogue across the disciplines for future collaborations. We include initial work that was done in preparation for the workshop, links to keynote and other presentations, and a summary of the findings, outcomes, and recommendations that were generated by the participants. Findings of the study include-the need for more extensive interdisciplinary interaction, identification of basic taxonomies and research issues, social informatics, establishment of a small number of common application domains, and field experience for members of the HRI community. An overall conclusion of the workshop was expressed as the following-HRI is a cross-disciplinary area, which poses barriers to meaningful research, synthesis, and technology transfer. The vocabularies, experiences, methodologies, and metrics of the communities are sufficiently different that cross-disciplinary research is unlikely to happen without sustained funding and an infrastructure to establish a new HRI community.