Human-robot Interaction Research Articles

Stretchable Self-Powered TENG Sensor Array for Human-Robot Interaction Based on Conductive Ionic Gels and LSTM Neural Network

Stretchable Self-Powered TENG Sensor Array for Human-Robot Interaction Based on Conductive Ionic Gels and LSTM Neural Network

Crucial hurdles to achieving human-robot harmony

Holistic consideration of the human and the robot is necessary to overcome hurdles in human-robot interaction.

Compassionate Care with Autonomous AI Humanoid Robots in Future Healthcare Delivery: A Multisensory Simulation of Next-Generation Models

The integration of AI and robotics in healthcare raises concerns, and additional issues regarding autonomous systems are anticipated. Effective communication is crucial for robots to be seen as “caring”, necessitating advanced mechatronic design and natural language processing (NLP). This paper examines the potential of humanoid robots to autonomously replicate compassionate care. The study employs computational simulations using mathematical and agent-based modeling to analyze human–robot interactions (HRIs) surpassing Tetsuya Tanioka’s TRETON. It incorporates stochastic elements (through neuromorphic computing) and quantum-inspired concepts (through the lens of Martha Rogers’ theory), running simulations over 100 iterations to analyze complex behaviors. Multisensory simulations (visual and audio) demonstrate the significance of “dynamic communication”, (relational) “entanglement”, and (healthcare system and robot’s function) “superpositioning” in HRIs. Quantum and neuromorphic computing may enable humanoid robots to empathetically respond to human emotions, based on Jean Watson’s ten caritas processes for creating transpersonal states. Autonomous AI humanoid robots will redefine the norms of “caring”. Establishing “pluralistic agreements” through open discussions among stakeholders worldwide is necessary to align innovations with the values of compassionate care within a “posthumanist” framework, where the compassionate care provided by Level 4 robots meets human expectations. Achieving compassionate care with autonomous AI humanoid robots involves translating nursing, communication, computer science, and engineering concepts into robotic care representations while considering ethical discourses through collaborative efforts. Nurses should lead the design and implementation of AI and robots guided by “technological knowing” in Rozzano Locsin’s TCCN theory.

An analysis of dialogue repair in virtual assistants

Conversational user interfaces have transformed human-computer interaction by providing nearly real-time responses to queries. However, misunderstandings between the user and system persist. This study explores the significance of interactional language in dialogue repair between virtual assistants and users by analyzing interactions with Google Assistant and Siri in both English and Spanish, focusing on the assistants’ utilization and response to the colloquial other-initiated repair strategy “huh?”, which is prevalent as a human-human dialogue repair strategy. Findings revealed ten distinct assistant-generated repair strategies, but an inability to replicate human-like strategies such as “huh?”. Despite slight variations in user acceptability judgments among the two surveyed languages, results indicated an overall hierarchy of preference towards specific dialogue repair strategies, with a notable disparity between the most preferred strategies and those frequently used by the assistants. These findings highlight discrepancies in how interactional language is utilized in human-computer interaction, underscoring the need for further research on the impact of interactional elements among different languages to advance the development of conversational user interfaces across domains, including within human-robot interaction.

Beyond Binary Dialogues: Research and Development of a Linguistically Nuanced Conversation Design for Social Robots in Group–Robot Interactions

In this paper, we detail the technical development of a conversation design that is sensitive to group dynamics and adaptable, taking into account the subtleties of linguistic variations between dyadic (i.e., one human and one agent) and group interactions in human–robot interaction (HRI) using the German language as a case study. The paper details the implementation of robust person and group detection with YOLOv5m and the expansion of knowledge databases using large language models (LLMs) to create adaptive multi-party interactions (MPIs) (i.e., group–robot interactions (GRIs)). We describe the use of LLMs to generate training data for socially interactive agents including social robots, as well as a self-developed synthesis tool, knowledge expander, to accurately map the diverse needs of different users in public spaces. We also outline the integration of a LLM as a fallback for open-ended questions not covered by our knowledge database, ensuring it can effectively respond to both individuals and groups within the MPI framework.

Human Perception of the Emotional Expressions of Humanoid Robot Body Movements: Evidence from Survey and Eye-Tracking Measurements

The emotional expression of body movement, which is an aspect of emotional communication between humans, has not been considered enough in the field of human–robot interactions (HRIs). This paper explores human perceptions of the emotional expressions of humanoid robot body movements to study the emotional design of the bodily expressions of robots and the characteristics of the human perception of these emotional body movements. Six categories of emotional behaviors, including happiness, anger, sadness, surprise, fear, and disgust, were designed by imitating human emotional body movements, and they were implemented on a Yanshee robot. A total of 135 participants were recruited for questionnaires and eye-tracking measurements. Statistical methods, including K-means clustering, repeated analysis of variance (ANOVA), Friedman’s ANOVA, and Spearman’s correlation test, were used to analyze the data. According to the statistical results of emotional categories, intensities, and arousals perceived by humans, a guide to grading the designed robot’s bodily expressions of emotion is created. By combining this guide with certain objective analyses, such as fixation and trajectory of eye movements, the characteristics of human perception, including the perceived differences between happiness and negative emotions and the trends of eye movements for different emotional categories, are described. This study not only illustrates subjective and objective evidence that humans can perceive robot bodily expressions of emotions through only vision but also provides helpful guidance for designing appropriate emotional bodily expressions in HRIs.

Attention-based hand pose estimation with voting and dual modalities

Hand pose estimation has recently emerged as a compelling topic in the robotic research community, because of its usefulness in learning from human demonstration or safe human–robot interaction. Although deep learning-based methods have been introduced for this task and have shown promise, it remains a challenging problem. To address this, we propose a novel end-to-end architecture for hand pose estimation using red-green-blue (RGB) and depth (D) data (RGB-D). Our approach processes the two data sources separately and utilizes a dense fusion network with an attention module to extract discriminative features. The features extracted include both spatial information and geometric constraints, which are fused to vote for the hand pose. We demonstrate that our voting mechanism in conjunction with the attention mechanism is particularly useful for solving the problem, especially when hands are heavily occluded by objects or are self-occluded. Our experimental results on benchmark datasets demonstrate that our approach outperforms state-of-the-art methods by a significant margin.

Commentary: The Future of Human-Robot Interactions

Service robots are poised for enormous impact helping to achieve the U.N Sustainable Development goals as well as supporting older adults and young learners. This article briefly explores these application areas in three invited commentaries written by leading scholars and an industry entrepreneur, expressing their varied perspectives on the future of service robots. The article accompanies this Journal of Service Research special issue on human-robot interactions. In their commentaries, the authors share views on areas of personal interest ranging from global grand challenges to healthy aging and educational development. Included within each commentary are managerial insights and suggestions for needed research in the highlighted area.

EXPRESS: ALTERCENTRISM IN PERSPECTIVE-TAKING: THE ROLE OF HUMANIZATION IN EMBODYING THE AGENT'S POINT OF VIEW.

We investigated the role of humanization in Visual Perspective-Taking (VPT) by testing whether and how agent's human-likeness and attractiveness ('hedonic quality') interact with social cues (action and eye gaze) in influencing the participants' disposition to embody another's perspective. In a VPT task, participants viewed scenes displaying an actor (human or robotic) grasping, gazing (or both) a target object, or adopting a still posture, and were required to judge the left/right location of the target, without receiving any instruction on the perspective to be assumed. Across two studies we selected human and robotic agents to use as actors in the VPT task. Results consistently demonstrated that participants could be effectively clustered by a data-driven method into two perspective-taking styles, depending on the presence of a systematic tendency to locate the target object in the VPT scenarios from own (egocentric) or the actor's (altercentric) point of view. The human vs. non-human nature of the agent seemed able to affect the participants' egocentric or altercentric tendency whereas both the agent's hedonic quality and social cues were not able to influence this propensity. Identifying the factors influencing altercentrism during human-robot interactions can be essential for developing artificial agents favouring user's acceptance and willingness to interact. In this respect, considering differences among individuals in their propensity to take another's point of view may be of central importance. Clustering approaches can represent a useful means to capture interindividual differences in this central aspect of human social cognition.

I Care That You Don’t Share: Confidentiality in Student-Robot Interactions

Enabled by technological advances, robot teachers have entered educational service frontlines. Scholars and policymakers suggest that during Human-Robot Interaction (HRI), human teachers should remain “in-the-loop” (i.e., oversee interactions between students and robots). Drawing on impression management theory, we challenge this belief to argue that robot teacher confidentiality (i.e., robot teachers not sharing student interactions with the human teacher) lets students make more use of the technology. To examine this effect and provide deeper insights into multiple mechanisms and boundary conditions, we conduct six field, laboratory and online experiments that use virtual and physical robot teachers (Total N = 2,012). We first show that students indeed make more use of a confidential (vs. nonconfidential) robot teacher (both physical and virtual). In a qualitative study (Study 2), we use structural topic modeling to inductively identify relevant mediators and moderators. Studies 3 through 5 provide support for these, showing two key mediators (i.e., social judgment concern and interaction anxiety) and two moderators (i.e., student prevention focus and teacher benevolence) for the effect of robot teacher confidentiality. Collectively, the present research introduces the concept of service robot confidentiality, illustrating why and how not sharing HRI with a third actor critically impacts educational service encounters.

Natural Language Processing in Robotics: Leveraging Python for Human-Robot Interaction

Natural Language Processing in Robotics: Leveraging Python for Human-Robot Interaction

Bending and sensing performances of electro-ionic soft actuators based on carboxylated cellulose nanofibers reinforced with graphene nanoplatelets

Abstract Soft robots not only possess greater degrees of freedom and the capability for continuous transformation, but they also offer exceptionally high safety in human-robot interactions, avoiding harm to the human body. Soft actuators are essential for developing high-performance soft robots, offering significant bending deformation, rapid response times, and prolonged operational capabilities. Herein, we present an ionic electroactive soft actuator based on functional cellulose nanofibers, graphene nanoplatelets, and ionic liquid. The proposed actuator achieved a large displacement about ±8 mm under 2.0 V at 0.1 Hz, with long working stability (98% of initial peak displacement maintained after 1260 cycles of cycling). The human-robot interaction applications of this actuator were explored by simulating human fingers. More importantly, the static and dynamic sensing performances of the actuator were investigated, finding that it generated a sensing voltage of 0.37 V at a vibration displacement of only 1.75 mm. The designed actuator provides a promising approach for developing high-performance soft robots, soft actuators, flexible sensors, and flexible active devices.

Textile Fabric Defect Detection Using Enhanced Deep Convolutional Neural Network with Safe Human–Robot Collaborative Interaction

The emergence of modern robotic technology and artificial intelligence (AI) enables a transformation in the textile sector. Manual fabric defect inspection is time-consuming, error-prone, and labor-intensive. This offers a great possibility for applying more AI-trained automated processes with safe human–robot interaction (HRI) to reduce risks of work accidents and occupational illnesses and enhance the environmental sustainability of the processes. In this experimental study, we developed, implemented, and tested a novel algorithm that detects fabric defects by utilizing enhanced deep convolutional neural networks (DCNNs). The proposed method integrates advanced DCNN architectures to automatically classify and detect 13 different types of fabric defects, such as double-ends, holes, broken ends, etc., ensuring high accuracy and efficiency in the inspection process. The dataset is created through augmentation techniques and a model is fine-tuned on a large dataset of annotated images using transfer learning approaches. The experiment was performed using an anthropomorphic robot that was programmed to move above the fabric. The camera attached to the robot detected defects in the fabric and triggered an alarm. A photoelectric sensor was installed on the conveyor belt and linked to the robot to notify it about an impending fabric. The CNN model architecture was enhanced to increase performance. Experimental findings show that the presented system can detect fabric defects with a 97.49% mean Average Precision (mAP).

Safety in Wearable Robotic Exoskeletons: Design, Control, and Testing Guidelines

Abstract Exoskeletons, wearable robotic devices designed to enhance human strength and endurance, find applications in various fields such as healthcare and industry; however, stringent safety measures should be adopted in such settings. This paper presents a comprehensive exploration of challenges associated with exoskeleton technology, ranging from mechanical issues to regulatory and ethical considerations. The enumerated challenges include joint hyper-extension or flexion, rapid or sudden motion, misalignment, fit, and comfort issues, mechanical failure, weight and mobility limitations, environmental challenges, power supply issues, high energy consumption and regeneration, fall risk or stability concerns, sensor failures, control algorithm malfunctions, machine-learning model challenges, communication disconnection, actuator malfunctions, unexpected human–robot interactions, and regulatory and ethical considerations. The paper outlines possible risks and suggests practical solutions based on design, control, and testing methods for each challenge. The objective is to offer a guideline for developers and users, emphasizing safety, reliability, and optimal performance in the ever-evolving landscape of exoskeleton technology. The guideline covers preoperation checks, user training, emergency response, real-time monitoring, and user interaction to ensure responsible innovation and user-centricity in exoskeleton development and deployment.

Transfer-Learning-Based Gesture and Pose Recognition System for Human–Robot Interaction: An Internet of Things Application

Transfer-Learning-Based Gesture and Pose Recognition System for Human–Robot Interaction: An Internet of Things Application

Adaptive FPGA-Based Accelerators for Human–Robot Interaction in Indoor Environments

This study addresses the challenges of human–robot interactions in real-time environments with adaptive field-programmable gate array (FPGA)-based accelerators. Predicting human posture in indoor environments in confined areas is a significant challenge for service robots. The proposed approach works on two levels: the estimation of human location and the robot’s intention to serve based on the human’s location at static and adaptive positions. This paper presents three methodologies to address these challenges: binary classification to analyze static and adaptive postures for human localization in indoor environments using the sensor fusion method, adaptive Simultaneous Localization and Mapping (SLAM) for the robot to deliver the task, and human–robot implicit communication. VLSI hardware schemes are developed for the proposed method. Initially, the control unit processes real-time sensor data through PIR sensors and multiple ultrasonic sensors to analyze the human posture. Subsequently, static and adaptive human posture data are communicated to the robot via Wi-Fi. Finally, the robot performs services for humans using an adaptive SLAM-based triangulation navigation method. The experimental validation was conducted in a hospital environment. The proposed algorithms were coded in Verilog HDL, simulated, and synthesized using VIVADO 2017.3. A Zed-board-based FPGA Xilinx board was used for experimental validation.

Multimodal flexible electronic skin with adjustable module size for robot collision detection

Purpose This paper aims to present a modular multimodal flexible electronic skin that can be used for robot collision detection in human–robot interactions. This type of electronic skin will meet the requirements of performance indicators such as sensing mode, sensing domain coverage and dynamic data update rate in human–robot interactions. Design/methodology/approach The electronic skin uses a modular architecture, and the sensing module is designed to be adjustable in size so that it can be easily deployed on complex robot surfaces, increasing area coverage, reducing power consumption, and improving data update rates. Findings The authors evaluated electronic skin through experiments using a UR5 robot. Electronic skin has high static scene perception differentiation and dynamic scene perception abilities. Moreover, the robot realizes a high-speed collision response based on the electronic skin proposed in this study. Originality/value The proposed electronic skin provides crucial technical support for advancing robotic technologies, and holds promising prospects for industrial applications.

I am frail and my robot is stuck on the corridor: cohabitation of people with frailty and robots

Abstract UK and European policies promote research and innovation for people living with frailty (PLWF), as realistic strategies for independent living. Yet homes are not designed for PLWF or robots, even though robots could support PLWF with activities, i.e., exercise and companionship. Adoption of robots in homes is limited to devices such as automated vacuum cleaners and voice assistants. Robotics aimed at assisting PLWF have been thus far been tested in laboratories and the study of real homes for the co-habitation of robots and older people is often neglected. We draw on expertise from human-robot interaction, built environment, public health and clinical practice to facilitate optimisation of human and robot cohabitation for PLWF. For this, we have reviewed different robots-for-frailty, in terms of, ability to fit and move within homes. We utilised a range of methods and sources of information including meetings with allied health professionals, visited the Robot House and explored the available robots from a fit for purpose perspective, audited five flats for PLWF in Gloucester, England in accordance with the needs of those with frailty, simulated robot’s function within the home environment, using persona’s for different use case scenarios, trial testing of selected robots in the home environment for comparison with simulation, using knowledge gained to extrapolate this data to estimate suitability for frail individuals in their homes. Most robots currently available would not adequately fit in an independent living accommodation or the smaller ones could be a tripping hazard. The communal areas of independent living would be more realistic for such human robot interactions, rather than the actual households. The research informs the process facilitating the adoption of robots to benefit PLWF both feasible and desired. The research produced a framework that can lead to future research exploring robot and PLWF co-habitation in real home settings. Key messages • The cohabitation of humans and robots currently faces barriers relevant to independent living built-environment spatial restrictions. • Technologies that support independent living and frailty must develop in spatially-unobtrusive ways.

Digital-twin co-simulation framework to support informed decision in healthcare planning and management

This manuscript aims to show the proof-of-concept for the integration of digital twin (DT) and discrete event co-simulation through its application to forward simulation of physical activities of care providers and robots in an intensive care unit (ICU). Co-simulation focuses on modular integration of heterogeneous simulation components to capture the complexity inherent in systems. Advancing this conception, the proposed co-simulation attempted to narrow gaps with real-world systems by using DT while simultaneously leveraging the forward simulation capacity of discrete event simulation (DES). In particular, we created a co-simulation for human–robot interactions in a simulated ICU facility, which modeled the hospital layout including the facility, robot, avatars for healthcare professionals, equipment, patient bed, and vital signs, which comprise physical twin (PT). In addition, specific patient-care scenarios were defined to drive PT for which the states of interest were represented in the DT, and then used for a dynamic input to the discrete event co-simulation. At the onset of the co-simulation, the parameters needed to run DES were manually configured to align the parameters with those of the DT. For internal validation, this DES was replicated under different work-system policies, and the co-simulation outcomes were interpreted in clinical context. Our co-simulation approach has potential for application to a broader class of problems in healthcare planning and management by supporting informed decision-making. Methodologically, timely information exchanges of PT with DT, and ensuing model updates in the Discrete Event System Specification (DEVS) is a promising approach for co-simulation, to maintain robustness against perturbations to real-world systems.

Human–robot interaction: predicting research agenda by long short-term memory

The article addresses the identification and prediction of research topics in human–robot interaction (HRI), fundamental in Industry 4.0 (I4.0) and future Industry 5.0 (I5.0). In the absence of research agendas in the scientific literature, the study proposes a multilayered model to create a precise agenda to guide the scientific community in new developments in collaborative robotics and HRI technologies. The methodology is divided into four stages, which make up the three layers of the model. In the first two stages, scientific articles on HRI for the period 2020–2021 were collected and analyzed using data mining techniques together with VantagePoint and Gephi software to identify keywords and their relationships. These initial stages form layer 1 of the model, where the main scientific themes are recognized. In the third stage, article titles and abstracts are cleaned and processed using natural language processing (NLP) techniques, generating word embeddings models that highlight relevant HRI-related terms, forming layer 2. The fourth and final stage uses Recurrent Neural Networks (RNN) with long short-term memory (LSTM) architecture to predict future topics, consolidating the previously identified terms and forming layer 3 of the model. The results show that in layer 1 HRI has intensive application in various sectors through advanced computational algorithms, with trust as a key feature. In layer 2, terms such as vision, sensors, communication, collaboration and anthropomorphic aspects are fundamental, while layer 3 anticipates future topics such as design, performance, method and controllers, essential to improve robot interaction. The study concludes that the methodology is effective in defining a robust and relevant research agenda. By identifying future trends and needs, this work fills a gap in the scientific literature, providing a valuable tool for the research community in the field of HRI.