Social Human-robot Interaction Research Articles

Cobots, Complexities, and Campfire: Report on the 4th HRI Summer School

The 4th Summer School on Social Human-Robot Interaction in Chȩciny, Poland, sponsored by IEEE RAS and the EU’s Horizon 2020, assembled nearly 100 professionals and students from various fields to explore human–robot communication. This year’s edition emphasized the human–human interactions in which human–robot interactions are embedded. On this background participants addressed essential themes such as robots’ agency, gestures, speech, and likeness to humans and the program featured expert lectures, practical workshops, and analysis sessions, fostering an interdisciplinary approach to learning. The successful integration of educational content and community building was evident in the positive testimonials, highlighting the importance of in-person exchanges and the combination of diverse views for progress in HRI research. This paper reflects on the accomplishments of the summer school, illustrating how it became a crucible for thought leadership and collaborative innovation in the HRI space.

An intelligent socially assistive robot-wearable sensors system for personalized user dressing assistance

Individuals living with cognitive impairments are faced with unique challenges in completing important activities of daily living such as dressing. In this paper, we present the first socially assistive robot-wearable sensors system to provide dressing assistance through social human-robot interactions. A novel robot-wearable architecture has been developed to recognize and classify user dressing actions and provide personalized prompts and feedback. Our system uses smart clothing with embedded strain sensors to estimate the user’s actions, which are then classified into different dressing steps. Our assistive robot uses a MAXQ hierarchical learning method to learn appropriate assistive behaviors to aid a user with the sequence of dressing steps. Experiments conducted validated the performance of our robot-wearable system in identifying and effectively responding to a variety of user states and dressing step actions. Furthermore, a robot demonstration study with stakeholders found that overall, they had positive perceptions and attitudes towards the socially assistive robot-wearable system, in particular with respect to its usefulness with the intended user population.

Determinants of Attitude to a Humanoid Social Robot in Care for Older Adults: A Post-Interaction Study.

BACKGROUND While there is a growing body of research examining opinions on social robots in elderly care, there is a lack of comprehensive studies investigating the underlying factors influencing these opinions. The Godspeed Questionnaire Series (GQS) measures perceptions related to human-robot interactions (HRIs). The Comprehensive Geriatric Assessment (CGA) is widely used to evaluate physical, cognitive, and social functions of older patients. The EASYCare 2010 Standard (EC) is a tool for assessing unmet needs in older individuals. TIAGo, a social humanoid robot, integrates perception, navigation, and HRI capabilities. This study aimed to identify the determinants of perception following interactions between older individuals and TIAGo, utilizing the GQS, selected CGA items, and EC. MATERIAL AND METHODS We analyzed a database of opinions from older individuals who interacted with TIAGo, based on the Users' Needs, Requirements, and Abilities Questionnaire. We examined the relationships between the robot's roles (companion/assistant/useful device), its assistive/social functions, and various characteristics of the older participants. RESULTS The study included 161 participants (mean age: 75.2±9.8 years), comprising 89 women and 113 institutionalized individuals. Positive correlations were observed between the robot's role, its functions, and the participants'; perceptions across most evaluated parameters (Anthropomorphism, Animacy, Likeability, Perceived intelligence, Perceived safety). Only a few individual correlations were found for other parameters. CONCLUSIONS The primary determinant of older individuals' opinions was their perception of the robot. Therefore, involving older adults in the co-design process of such robots is crucial. Additionally, a paradigm shift is needed in the study of humanoid social robots, focusing on successful aging rather than deficits associated with aging.

Heat-assisted detection and ranging.

Machine perception uses advanced sensors to collect information about the surrounding scene for situational awareness1-7. State-of-the-art machine perception8 using active sonar, radar and LiDAR to enhance camera vision9 faces difficulties when the number of intelligent agents scales up10,11. Exploiting omnipresent heat signal could be a new frontier for scalable perception. However, objects and their environment constantly emit and scatter thermal radiation, leading to textureless images famously known as the 'ghosting effect'12. Thermal vision thus has no specificity limited by information loss, whereas thermal ranging-crucial for navigation-has been elusive even when combined with artificial intelligence (AI)13. Here we propose and experimentally demonstrate heat-assisted detection and ranging (HADAR) overcoming this open challenge of ghosting and benchmark it against AI-enhanced thermal sensing. HADAR not only sees texture and depth through the darkness as if it were day but also perceives decluttered physical attributes beyond RGB or thermal vision, paving the way to fully passive and physics-aware machine perception. We develop HADAR estimation theory and address its photonic shot-noise limits depicting information-theoretic bounds to HADAR-based AI performance. HADAR ranging at night beats thermal ranging and shows an accuracy comparable with RGB stereovision in daylight. Our automated HADAR thermography reaches the Cramér-Rao bound on temperature accuracy, beating existing thermography techniques. Our work leads to a disruptive technology that can accelerate the Fourth Industrial Revolution (Industry 4.0)14 with HADAR-based autonomous navigation and human-robot social interactions.

Can robots be trustworthy?

Definition of the problemThis article critically addresses the conceptualization of trust in the ethical discussion on artificial intelligence (AI) in the specific context of social robots in care. First, we attempt to define in which respect we can speak of ‘social’ robots and how their ‘social affordances’ affect the human propensity to trust in human–robot interaction. Against this background, we examine the use of the concept of ‘trust’ and ‘trustworthiness’ with respect to the guidelines and recommendations of the High-Level Expert Group on AI of the European Union.ArgumentsTrust is analyzed as a multidimensional concept and phenomenon that must be primarily understood as departing from trusting as a human functioning and capability. To trust is an essential part of the human basic capability to form relations with others. We further want to discuss the concept of responsivity which has been established in phenomenological research as a foundational structure of the relation between the self and the other. We argue that trust and trusting as a capability is fundamentally responsive and needs responsive others to be realized. An understanding of responsivity is thus crucial to conceptualize trusting in the ethical framework of human flourishing. We apply a phenomenological–anthropological analysis to explore the link between certain qualities of social robots that construct responsiveness and thereby simulate responsivity and the human propensity to trust.ConclusionAgainst this background, we want to critically ask whether the concept of trustworthiness in social human–robot interaction could be misguided because of the limited ethical demands that the constructed responsiveness of social robots is able to answer to.

A Trained Humanoid Robot can Perform Human-Like Crossmodal Social Attention and Conflict Resolution

To enhance human-robot social interaction, it is essential for robots to process multiple social cues in a complex real-world environment. However, incongruency of input information across modalities is inevitable and could be challenging for robots to process. To tackle this challenge, our study adopted the neurorobotic paradigm of crossmodal conflict resolution to make a robot express human-like social attention. A behavioural experiment was conducted on 37 participants for the human study. We designed a round-table meeting scenario with three animated avatars to improve ecological validity. Each avatar wore a medical mask to obscure the facial cues of the nose, mouth, and jaw. The central avatar shifted its eye gaze while the peripheral avatars generated sound. Gaze direction and sound locations were either spatially congruent or incongruent. We observed that the central avatar’s dynamic gaze could trigger crossmodal social attention responses. In particular, human performance was better under the congruent audio-visual condition than the incongruent condition. Our saliency prediction model was trained to detect social cues, predict audio-visual saliency, and attend selectively for the robot study. After mounting the trained model on the iCub, the robot was exposed to laboratory conditions similar to the human experiment. While the human performance was overall superior, our trained model demonstrated that it could replicate attention responses similar to humans.

Editorial Introduction to Special Issue on Machine Learning for Social Human-Robot Interaction

Editorial Introduction to Special Issue on Machine Learning for Social Human-Robot Interaction

SoftSAR: The New Softer Side of Socially Assistive Robots-Soft Robotics with Social Human-Robot Interaction Skills.

When we think of "soft" in terms of socially assistive robots (SARs), it is mainly in reference to the soft outer shells of these robots, ranging from robotic teddy bears to furry robot pets. However, soft robotics is a promising field that has not yet been leveraged by SAR design. Soft robotics is the incorporation of smart materials to achieve biomimetic motions, active deformations, and responsive sensing. By utilizing these distinctive characteristics, a new type of SAR can be developed that has the potential to be safer to interact with, more flexible, and uniquely uses novel interaction modes (colors/shapes) to engage in a heighted human-robot interaction. In this perspective article, we coin this new collaborative research area as SoftSAR. We provide extensive discussions on just how soft robotics can be utilized to positively impact SARs, from their actuation mechanisms to the sensory designs, and how valuable they will be in informing future SAR design and applications. With extensive discussions on the fundamental mechanisms of soft robotic technologies, we outline a number of key SAR research areas that can benefit from using unique soft robotic mechanisms, which will result in the creation of the new field of SoftSAR.

The Influence of Visible Cables and Story Content on Perceived Autonomy in Social Human–Robot Interaction

From teaching technical skills to telling bedtime stories, social robots support various edutainment tasks that require smooth communication. Previous studies often emphasized the importance of the autonomy of social robots for those tasks. However, the cabling of robots with power sources and/ or host computers is often required due to technical restrictions. However, it is currently unclear if the cabling of robots makes a difference in perceived autonomy. Therefore, this study examined the influence of visible cables in different tasks on the perception of a social robot. In an online survey, participants evaluated videos of a social robot that was either equipped with a cable or not and told either a story with technical educational content or socially entertaining content. No significant differences were revealed between the cabled and the non-cabled robot, neither for the perceived autonomy nor for the associated concepts of the Godspeed questionnaire series. In addition, the story content did not influence perceived autonomy. However, the robot that told the technical content was perceived as significantly more intelligent and tended to be perceived as more likable than the robot that told the social content. Moreover, the interaction effect of cabling and story content for perceived safety just failed to reach the conventional level of significance. In the social content condition, the non-cabled robot tended to be perceived as less safe than the cabled robot. This was not true for the technical content condition. In conclusion, the results showed the importance of considering story content. Due to methodological limitations of the current study, namely, the lack of gestures accompanying the storytelling and the video-based approach, the missing effect of cabling in regard to perceived autonomy should be investigated in the future via real-life interaction studies.

Society 5.0: Internet as if People Mattered

While the primary focus of 5G has been on industry verticals, future 6G mobile networks are anticipated to become more human-centered. Emerging cyber-physical-social systems (CPSSs) aim at functionally integrating human beings into today's cyber-physical systems at the social, cognitive, and physical levels. CPSSs are instrumental in realizing the human-centered Society 5.0 vision. Society 5.0 envisions human beings increasingly interacting with social robots and embodied artificial intelligence in their daily lives. In this article, we build on our recent work on robonomics in the 6G era. Robonomics is an emerging field that investigates social human-robot interaction and its sociotechnical impact as well as blockchain technologies and cryptocurrencies, not only coins but - more interestingly - also tokens. Specifically, we study the tokenization process of creating tokenized digital twins of assets and access rights in the physical and digital world, paying close attention to its central role in ushering in the future Web3 and its underlying token economy, the successor of today's information and platform economies. After introducing our CPSS-based bottom-up multilayer token engineering framework for Society 5.0, we experimentally demonstrate how the collective human intelligence of a blockchain-enabled decentralized autonomous organization can be enhanced via purpose-driven tokens.

A Framework to Study and Design Communication with Social Robots

Communication is a central component in social human–robot interaction that needs to be planned and designed prior to the actual communicative act. We therefore propose a pragmatic, linear view of communication design for social robots that corresponds to a sender–receiver perspective. Our framework is based on Lasswell’s 5Ws of mass communication: Who, says what, in which channel, to whom, with what effect. We extend and adapt this model to communication in HRI. In addition, we point out that, besides the predefined communicative acts of a robot, other characteristics, such as a robot’s morphology, can also have an impact on humans, since humans tend to assign meaning to every cue in robots’ behavior and appearance. We illustrate the application of the extended framework to three different studies on human–robot communication to demonstrate the incremental value as it supports a systematic evaluation and the identification of similarities, differences, and research gaps. The framework therefore offers the opportunity for meta-analyses of existing research and additionally draws the path for future robust research designs for studying human–robot communication.

Training Socially Engaging Robots: Modeling Backchannel Behaviors with Batch Reinforcement Learning

A key aspect of social human-robot interaction is natural non-verbal communication. In this work, we train an agent with batch reinforcement learning to generate nods and smiles as backchannels in order to increase the naturalness of the interaction and to engage humans. We introduce the Sequential Random Deep Q-Network (SRDQN) method to learn a policy for backchannel generation, that explicitly maximizes user engagement. The proposed SRDQN method outperforms the existing vanilla Q-learning methods when evaluated using off-policy policy evaluation techniques. Furthermore, to verify the effectiveness of SRDQN, a human-robot experiment has been designed and conducted with an expressive 3d robot head. The experiment is based on a story-shaping game designed to create an interactive social activity with the robot. The engagement of the participants during the interaction is computed from user's social signals like backchannels, mutual gaze and adjacency pair. The subjective feedback from participants and the engagement values strongly indicate that our framework is a step forward towards the autonomous learning of a socially acceptable backchanneling behavior.

Envisioning social drones in education.

Education is one of the major application fields in social Human-Robot Interaction. Several forms of social robots have been explored to engage and assist students in the classroom environment, from full-bodied humanoid robots to tabletop robot companions, but flying robots have been left unexplored in this context. In this paper, we present seven online remote workshops conducted with 20 participants to investigate the application area of Education in the Human-Drone Interaction domain; particularly focusing on what roles a social drone could fulfill in a classroom, how it would interact with students, teachers and its environment, what it could look like, and what would specifically differ from other types of social robots used in education. In the workshops we used online collaboration tools, supported by a sketch artist, to help envision a social drone in a classroom. The results revealed several design implications for the roles and capabilities of a social drone, in addition to promising research directions for the development and design in the novel area of drones in education.

Android as a Receptionist in a Shopping Mall Using Inverse Reinforcement Learning

For human-robot interaction (HRI), it is difficult to hand-craft all the rules for robots owing to diverse situations. Therefore, inverse reinforcement learning (IRL) is a potential solution that helps transfer human knowledge about interactions to robots. However, the feasibility of practically using IRL for HRI remains unknown. Here, we demonstrate a practical HRI application of IRL. An android was trained using IRL and acted as a receptionist to encourage visitors to practice hand hygiene in a shopping mall. We found that android learning through IRL has a competitive ability to a well-trained human operator on the reception task. Furthermore, we found that the android maintained high performance regardless of customer traffic. Our results demonstrate the potential of IRL in advancing the social HRI field. We anticipate that our work will be a starting point for using IRL in future HRI applications.

Novelty Knows No Boundaries: Why a Proper Investigation of Novelty Effects Within SHRI Should Begin by Addressing the Scientific Plurality of the Field.

Research on psychological novelty effects within the fields of Social Robotics and Human-Robot Interaction (together: SHRI) so far has failed to gather the momentum it deserves. With the aid of exemplary descriptions of how psychological novelty is currently approached and researched across (certain main regions of) the larger scientific landscape, I argue that the treatment of novelty effects within the multidisciplinary SHRI reflects larger circumstances of fragmentation and heterogeneity in novelty research in general. I further propose that while the concept of novelty may currently function as a Boundary Object between the contributing domains of SHRI, a properly integrated, interdisciplinary concept of novelty is needed in order to capture and investigate the scope and scale of novelty effects within research on social human-robot interaction. Building on research on the New Ontological Category Hypothesis and related studies, I argue that the novelty of social robots can be understood as radical to the extent that their comprehension requires revisions of traditional core categories of being. In order to investigate the sui generis effects of such novelty, which should not be narrowly understood as mere “noise” in the data, it is paramount that the field of SHRI begin by working out a shared, integrative framework of psychological novelty and novelty effects.

Self-Explaining Social Robots: An Explainable Behavior Generation Architecture for Human-Robot Interaction.

In recent years, the ability of intelligent systems to be understood by developers and users has received growing attention. This holds in particular for social robots, which are supposed to act autonomously in the vicinity of human users and are known to raise peculiar, often unrealistic attributions and expectations. However, explainable models that, on the one hand, allow a robot to generate lively and autonomous behavior and, on the other, enable it to provide human-compatible explanations for this behavior are missing. In order to develop such a self-explaining autonomous social robot, we have equipped a robot with own needs that autonomously trigger intentions and proactive behavior, and form the basis for understandable self-explanations. Previous research has shown that undesirable robot behavior is rated more positively after receiving an explanation. We thus aim to equip a social robot with the capability to automatically generate verbal explanations of its own behavior, by tracing its internal decision-making routes. The goal is to generate social robot behavior in a way that is generally interpretable, and therefore explainable on a socio-behavioral level increasing users' understanding of the robot's behavior. In this article, we present a social robot interaction architecture, designed to autonomously generate social behavior and self-explanations. We set out requirements for explainable behavior generation architectures and propose a socio-interactive framework for behavior explanations in social human-robot interactions that enables explaining and elaborating according to users' needs for explanation that emerge within an interaction. Consequently, we introduce an interactive explanation dialog flow concept that incorporates empirically validated explanation types. These concepts are realized within the interaction architecture of a social robot, and integrated with its dialog processing modules. We present the components of this interaction architecture and explain their integration to autonomously generate social behaviors as well as verbal self-explanations. Lastly, we report results from a qualitative evaluation of a working prototype in a laboratory setting, showing that (1) the robot is able to autonomously generate naturalistic social behavior, and (2) the robot is able to verbally self-explain its behavior to the user in line with users' requests.

Non-Smooth Control Barrier Navigation Functions for STL Motion Planning.

This paper reports on a new approach to Signal Temporal Logic (STL) control synthesis, that 1) utilizes a navigation function as the basis to construct a Control Barrier Function (CBF), and 2) composes navigation function-based barrier functions using nonsmooth mappings to encode Boolean operations between the predicates that those barrier functions encode. Because of these two key features, the reported approach 1) covers a larger fragment of STL compared to existing approaches, 2) alleviates the computational cost associated with evaluation of the control law for the system in existing STL control barrier function methodologies, and 3) simultaneously relaxes some of the conservativeness of smooth combinations of barrier functions as a means of implementing Boolean operators. The paper demonstrates the efficacy of this new approach with three simulation case studies, one aiming at illustrating how complex STL motion planning specification can be realized, the second highlights the less-conservativeness of the approach in comparison to the existing methods, and another that shows how this technology can be brought to bear to push the envelope in the context of human-robot social interaction.

Hybrid Hierarchical Learning for Adaptive Persuasion in Human-Robot Interaction

Adaptive learning is critical to helping robots personalize their interactions with people, particularly when considering skills needed by socially assistive robots, such as persuasion. In this letter, we propose a novel, hybrid hierarchical learning architecture for use in social human-robot interaction (HRI) to adapt robot persuasive behaviors to both the static (e.g., need for cognition) and dynamic (e.g., affect) considerations of a user. A learning hierarchy is introduced that uses a contextual bandit approach in the top level to optimize for a static cognition bias and <i>Q</i>-Learning in the lower level to optimize selection of a robot persuasive strategy to deploy that aligns with a user&#x0027;s affect. We compare the performance of our system with a non-hierarchical learning method in simulated experiments for the task of persuading people to do daily exercises. The results show that our hybrid hierarchical architecture outperforms a non-hierarchical benchmark in learning speed and robustness to both longitudinal user change and noisy observations. Our architecture is the first to: 1) persuasively adapt to different users during social HRI considering both static and dynamic user change, and 2) use user state decomposition in persuasive HRI.

Applying Service-Dominant Logic and Conversation Management Principles to Social Robotics for Autism Spectrum Disorder

Anthropomorphism in social robots amplifies the big five human personality traits (extraversion, conscientiousness, neuroticism, openness to experience, and agreeableness), consequently aiding with the social motivation needs of consumers (especially individuals with autism spectrum disorder or ASD). According to the social motivation theory of autism, consumers with ASD show deficits in orienting toward social stimuli, engaging with humans, and maintaining social relations. Robotic anthropomorphism has been found to be positively related to the big five human personality types and robot likeability in human-robot interaction (HRI) situations. This research focuses on the conversational approach of social robotics using service-dominant approach. Conversation-based perspectives have been studied extensively in organizational and management literature; however, these perspectives have not been utilized in the context of social robots, HRI, and autism.

Detecting Groups and Estimating F-Formations for Social Human–Robot Interactions

The ability of a robot to detect and join groups of people is of increasing importance in social contexts, and for the collaboration between teams of humans and robots. In this paper, we propose a framework, autonomous group interactions for robots (AGIR), that endows a robot with the ability to detect such groups while following the principles of F-formations. Using on-board sensors, this method accounts for a wide spectrum of different robot systems, ranging from autonomous service robots to telepresence robots. The presented framework detects individuals, estimates their position and orientation, detects groups, determines their F-formations, and is able to suggest a position for the robot to enter the social group. For evaluation, two simulation scenes were developed based on the standard real-world datasets. The 1st scene is built with 20 virtual agents (VAs) interacting in 7 different groups of varying sizes and 3 different formations. The 2nd scene is built with 36 VAs, positioned in 13 different groups of varying sizes and 6 different formations. A model of a Pepper robot is used in both simulated scenes in randomly generated different positions. The ability for the robot to estimate orientation, detect groups, and estimate F-formations at various locations is used to determine the validation of the approaches. The obtained results show a high accuracy within each of the simulated scenarios and demonstrates that the framework is able to work from an egocentric view with a robot in real time.