Robot Interaction Research Articles

Multimodal flexible electronic skin with adjustable module size for robot collision detection

Purpose This paper aims to present a modular multimodal flexible electronic skin that can be used for robot collision detection in human–robot interactions. This type of electronic skin will meet the requirements of performance indicators such as sensing mode, sensing domain coverage and dynamic data update rate in human–robot interactions. Design/methodology/approach The electronic skin uses a modular architecture, and the sensing module is designed to be adjustable in size so that it can be easily deployed on complex robot surfaces, increasing area coverage, reducing power consumption, and improving data update rates. Findings The authors evaluated electronic skin through experiments using a UR5 robot. Electronic skin has high static scene perception differentiation and dynamic scene perception abilities. Moreover, the robot realizes a high-speed collision response based on the electronic skin proposed in this study. Originality/value The proposed electronic skin provides crucial technical support for advancing robotic technologies, and holds promising prospects for industrial applications.

Digital-twin co-simulation framework to support informed decision in healthcare planning and management

This manuscript aims to show the proof-of-concept for the integration of digital twin (DT) and discrete event co-simulation through its application to forward simulation of physical activities of care providers and robots in an intensive care unit (ICU). Co-simulation focuses on modular integration of heterogeneous simulation components to capture the complexity inherent in systems. Advancing this conception, the proposed co-simulation attempted to narrow gaps with real-world systems by using DT while simultaneously leveraging the forward simulation capacity of discrete event simulation (DES). In particular, we created a co-simulation for human–robot interactions in a simulated ICU facility, which modeled the hospital layout including the facility, robot, avatars for healthcare professionals, equipment, patient bed, and vital signs, which comprise physical twin (PT). In addition, specific patient-care scenarios were defined to drive PT for which the states of interest were represented in the DT, and then used for a dynamic input to the discrete event co-simulation. At the onset of the co-simulation, the parameters needed to run DES were manually configured to align the parameters with those of the DT. For internal validation, this DES was replicated under different work-system policies, and the co-simulation outcomes were interpreted in clinical context. Our co-simulation approach has potential for application to a broader class of problems in healthcare planning and management by supporting informed decision-making. Methodologically, timely information exchanges of PT with DT, and ensuing model updates in the Discrete Event System Specification (DEVS) is a promising approach for co-simulation, to maintain robustness against perturbations to real-world systems.

I am frail and my robot is stuck on the corridor: cohabitation of people with frailty and robots

Abstract UK and European policies promote research and innovation for people living with frailty (PLWF), as realistic strategies for independent living. Yet homes are not designed for PLWF or robots, even though robots could support PLWF with activities, i.e., exercise and companionship. Adoption of robots in homes is limited to devices such as automated vacuum cleaners and voice assistants. Robotics aimed at assisting PLWF have been thus far been tested in laboratories and the study of real homes for the co-habitation of robots and older people is often neglected. We draw on expertise from human-robot interaction, built environment, public health and clinical practice to facilitate optimisation of human and robot cohabitation for PLWF. For this, we have reviewed different robots-for-frailty, in terms of, ability to fit and move within homes. We utilised a range of methods and sources of information including meetings with allied health professionals, visited the Robot House and explored the available robots from a fit for purpose perspective, audited five flats for PLWF in Gloucester, England in accordance with the needs of those with frailty, simulated robot’s function within the home environment, using persona’s for different use case scenarios, trial testing of selected robots in the home environment for comparison with simulation, using knowledge gained to extrapolate this data to estimate suitability for frail individuals in their homes. Most robots currently available would not adequately fit in an independent living accommodation or the smaller ones could be a tripping hazard. The communal areas of independent living would be more realistic for such human robot interactions, rather than the actual households. The research informs the process facilitating the adoption of robots to benefit PLWF both feasible and desired. The research produced a framework that can lead to future research exploring robot and PLWF co-habitation in real home settings. Key messages • The cohabitation of humans and robots currently faces barriers relevant to independent living built-environment spatial restrictions. • Technologies that support independent living and frailty must develop in spatially-unobtrusive ways.

Human–robot interaction: predicting research agenda by long short-term memory

The article addresses the identification and prediction of research topics in human–robot interaction (HRI), fundamental in Industry 4.0 (I4.0) and future Industry 5.0 (I5.0). In the absence of research agendas in the scientific literature, the study proposes a multilayered model to create a precise agenda to guide the scientific community in new developments in collaborative robotics and HRI technologies. The methodology is divided into four stages, which make up the three layers of the model. In the first two stages, scientific articles on HRI for the period 2020–2021 were collected and analyzed using data mining techniques together with VantagePoint and Gephi software to identify keywords and their relationships. These initial stages form layer 1 of the model, where the main scientific themes are recognized. In the third stage, article titles and abstracts are cleaned and processed using natural language processing (NLP) techniques, generating word embeddings models that highlight relevant HRI-related terms, forming layer 2. The fourth and final stage uses Recurrent Neural Networks (RNN) with long short-term memory (LSTM) architecture to predict future topics, consolidating the previously identified terms and forming layer 3 of the model. The results show that in layer 1 HRI has intensive application in various sectors through advanced computational algorithms, with trust as a key feature. In layer 2, terms such as vision, sensors, communication, collaboration and anthropomorphic aspects are fundamental, while layer 3 anticipates future topics such as design, performance, method and controllers, essential to improve robot interaction. The study concludes that the methodology is effective in defining a robust and relevant research agenda. By identifying future trends and needs, this work fills a gap in the scientific literature, providing a valuable tool for the research community in the field of HRI.

Modeling, Simulation, and Kinematic Validation of Transfemoral Prosthetic Mechanism With Ankle Varus–Valgus Characteristic

Abstract Inspired by the kinesiology of human bionic joints, a transfemoral prosthetic mechanism based on a functional structure of parallel mechanism is developed for the transfemoral amputees. The walking interactive simulation is implemented based on human-prosthesis modeling to verify the kinematics of the designed prosthetic mechanism, as well as to explore compatibility between the amputees and prosthesis. Then, simulation-based prosthetic optimization is performed to pursue an optimized human-prosthesis model with economic metabolic consumption while eliminating compatibility errors including the joints' misalignment error between the affected limb and healthy limb, and the assembly error between human and prosthesis, so that the potential physical health problems can be avoided efficiently. This method is valuable for the optimal design of interactive rehabilitation robots. Finally, a developed proportional-integral-derivative-based (PID-based) finite-state machine (FSM) strategy is used, and the kinematic validation is carried out. The results show that the designed prosthesis possesses ankle varus–valgus characteristic, and it has a high human-like motion accuracy due to the FSM control can track prosthetic motion in each gait event. What's more, the prosthetic optimization can be an efficient method to enhance the biomechanical performance of human-prosthetic model so that the amputees have a more natural and symmetry gait.

A Human-Centered View of Continual Learning: Understanding Interactions, Teaching Patterns, and Perceptions of Human Users Toward a Continual Learning Robot in Repeated Interactions

Continual learning (CL) has emerged as an important avenue of research in recent years, at the intersection of Machine Learning (ML) and Human–Robot Interaction (HRI), to allow robots to continually learn in their environments over long-term interactions with humans. Most research in CL, however, has been robot-centered to develop CL algorithms that can quickly learn new information on systematically collected static datasets. In this article, we take a human-centered approach to CL, to understand how humans interact with, teach, and perceive CL robots over the long term, and if there are variations in their teaching styles. We developed a socially guided CL system that integrates CL models for object recognition with a mobile manipulator robot and allows humans to directly teach and test the robot in real time over multiple sessions. We conducted an in-person study with 60 participants who interacted with the CL robot in 300 sessions with 5 sessions per participant. In this between-participant study, we used three different CL models deployed on a mobile manipulator robot. An extensive qualitative and quantitative analysis of the data collected in the study shows that there is significant variation among the teaching styles of individual users indicating the need for personalized adaptation to their distinct teaching styles. Our analysis shows that the constrained experimental setups that have been widely used to test most CL models are not adequate, as real users interact with and teach CL robots in a variety of ways. Finally, our analysis shows that although users have concerns about CL robots being deployed in our daily lives, they mention that with further improvements CL robots could assist older adults and people with disabilities in their homes.

Body Movement Mirroring and Synchrony in Human–Robot Interaction

This review article provides an overview of papers that have studied body movement mirroring and synchrony within the field of human-robot interaction. The papers included in this review cover system studies, which focus on evaluating the technical aspects of mirroring and synchrony robotic systems, and user studies, which focus on measuring particular interaction outcomes or attitudes towards robots expressing mirroring and synchrony behaviors. We review the papers in terms of the employed robotic platforms and the focus on parts of the body, the techniques used to sense and react to human motion, the evaluation methods, the intended applications of the human-robot interaction systems and the scenarios utilized in user studies. Finally, challenges and possible future directions are considered and discussed.

An Assessment of an Inpatient Robotic Nurse Assistant: A Mixed-Method Study

The worldwide nursing shortage has led to the exploration of using robotics to support care delivery and reduce nurses’ workload. In this observational, mixed-method study, we examined the implementation of a robotic nurse assistant (RNA) in a hospital ward to support vital signs measurements, medication, and item delivery. Human–robot interaction was assessed in four domains: usability, social acceptance, user experience, and its societal impact. Patients in a general medicine ward were recruited to participate in a one-time trial with the RNA and a post-trial 75-question survey. Patients’ interactions with the RNA were video recorded for analysis including patients’ behaviours, facial emotions, and visual attention. Focus group discussions with nurses elicited their perceptions of working with the RNA, areas for improvement, and scalability. Sixty-seven patients aged 21–79 participated in the trial. Eight in 10 patients reported positive interactions with the RNA. When the RNA did not perform to expectations, only 25% of patients attributed fault to the RNA. Video analysis showed patients at ease interacting with the RNA despite some technical problems. Nurses saw potential for the RNA taking over routine tasks. However, they were sceptical of real time savings and were concerned with the RNA’s ability to work well with older patients. Patients and nurses suggested greater interactivity between RNA and patients. Future studies should examine potential timesaving and whether time saved translated to nurses performing higher value clinical tasks. The utility of improved RNA’s social capability in a hospital setting should be explored as well.

THÖR-MAGNI: A large-scale indoor motion capture recording of human movement and robot interaction

We present a new large dataset of indoor human and robot navigation and interaction, called THÖR-MAGNI, that is designed to facilitate research on social human navigation: for example, modeling and predicting human motion, analyzing goal-oriented interactions between humans and robots, and investigating visual attention in a social interaction context. THÖR-MAGNI was created to fill a gap in available datasets for human motion analysis and HRI. This gap is characterized by a lack of comprehensive inclusion of exogenous factors and essential target agent cues, which hinders the development of robust models capable of capturing the relationship between contextual cues and human behavior in different scenarios. Unlike existing datasets, THÖR-MAGNI includes a broader set of contextual features and offers multiple scenario variations to facilitate factor isolation. The dataset includes many social human–human and human–robot interaction scenarios, rich context annotations, and multi-modal data, such as walking trajectories, gaze-tracking data, and lidar and camera streams recorded from a mobile robot. We also provide a set of tools for visualization and processing of the recorded data. THÖR-MAGNI is, to the best of our knowledge, unique in the amount and diversity of sensor data collected in a contextualized and socially dynamic environment, capturing natural human–robot interactions.

Learning representations for robust human–robot interaction

AbstractThis article summarizes the author's presentation in the New Faculty Highlight at the Thirty‐Eighth AAAI Conference on Artificial Intelligence. It discusses the desired properties of representations for enabling robust human–robot interaction. Examples from the author's work are presented to show how to build these properties into models for performing tasks with natural language guidance and engaging in social interactions with other agents.

A Two-Stage Facial Kinematic Control Strategy for Humanoid Robots Based on Keyframe Detection and Keypoint Cubic Spline Interpolation

A plentiful number of facial expressions is the basis of natural human–robot interaction for high-fidelity humanoid robots. The facial expression imitation of humanoid robots involves the transmission of human facial expression data to servos situated within the robot’s head. These data drive the servos to manipulate the skin, thereby enabling the robot to exhibit various facial expressions. However, since the mechanical transmission rate cannot keep up with the data processing rate, humanoid robots often suffer from jitters in the imitation. We conducted a thorough analysis of the transmitted facial expression sequence data and discovered that they are extremely redundant. Therefore, we designed a two-stage strategy for humanoid robots based on facial keyframe detection and facial keypoint detection to achieve more natural and smooth expression imitation. We first built a facial keyframe detection model based on ResNet-50, combined with optical flow estimation, which can identify key expression frames in the sequence. Then, a facial keypoint detection model is used on the keyframes to obtain the facial keypoint coordinates. Based on the coordinates, the cubic spline interpolation method is used to obtain the motion trajectory parameters of the servos, thus realizing the robust control of the humanoid robot’s facial expression. Experiments show that, unlike before where the robot’s imitation would stutter at frame rates above 25 fps, our strategy allows the robot to maintain good facial expression imitation similarity (cosine similarity of 0.7226), even at higher frame rates.

A Human–Security Robot Interaction Literature Review

As advances in robotics continue, security robots are increasingly integrated into public and private security, enhancing protection in locations such as streets, parks, and shopping malls. To be effective, security robots must interact with civilians and security personnel, underscoring the need to enhance our knowledge of their interactions with humans. To investigate this issue, the authors systematically reviewed 47 studies on human interaction with security robots, covering 2003 to 2023. Papers in this domain have significantly increased over the last seven years. The paper provides three contributions. First, it comprehensively summarizes existing literature on human interaction with security robots. Second, it employs the Human–Robot Integrative Framework (HRIF) to categorize this literature into three main thrusts: human, robot, and context. The framework is leveraged to derive insights into the methodologies, tasks, predictors, and outcomes studied. Last, the paper synthesizes and discusses the findings from the reviewed literature, identifying avenues for future research in this domain.

“You Scare Me”: The Effects of Humanoid Robot Appearance, Emotion, and Interaction Skills on Uncanny Valley Phenomenon

This study investigates the effects of humanoid robot appearance, emotional expression, and interaction skills on the uncanny valley phenomenon among university students using the social humanoid robot (SHR) Ameca. Two fundamental studies were conducted within a university setting: Study 1 assessed student expectations of SHRs in a hallway environment, emphasizing the need for robots to integrate seamlessly and engage effectively in social interactions; Study 2 compared the humanlikeness of three humanoid robots, ROMAN, ROBIN, and EMAH (employing the EMAH robotic system implemented on Ameca). The initial findings from corridor interactions highlighted a diverse range of human responses, from engagement and curiosity to indifference and unease. Additionally, the online survey revealed significant insights into expected non-verbal communication skills, continuous learning, and comfort levels during hallway conversations with robots. Notably, certain humanoid robots evoked stronger emotional reactions, hinting at varying degrees of humanlikeness and the influence of interaction quality. The EMAH system was frequently ranked as most humanlike before the study, while post-study perceptions indicated a shift, with EMAH and ROMAN showing significant changes in perceived humanlikeness, suggesting a re-evaluation by participants influenced by their interactive experiences. This research advances our understanding of the uncanny valley phenomenon and the role of humanoid design in enhancing human–robot interaction, marking the first direct comparison between the most advanced, humanlike research robots.

A Survey of Multimodal Perception Methods for Human–Robot Interaction in Social Environments

Human–robot interaction (HRI) in human social environments (HSEs) poses unique challenges for robot perception systems, which must combine asynchronous, heterogeneous data streams in real time. Multimodal perception systems are well-suited for HRI in HSEs and can provide more rich, robust interaction for robots operating among humans. In this article, we provide an overview of multimodal perception systems being used in HSEs, which is intended to be an introduction to the topic and summary of relevant trends, techniques, resources, challenges, and terminology. We surveyed 15 peer-reviewed robotics and HRI publications over the past 10+ years, providing details about the data acquisition, processing, and fusion techniques used in 65 multimodal perception systems across various HRI domains. Our survey provides information about hardware, software, datasets, and methods currently available for HRI perception research, as well as how these perception systems are being applied in HSEs. Based on the survey, we summarize trends, challenges, and limitations of multimodal human perception systems for robots, then identify resources for researchers and developers and propose future research areas to advance the field.

A Systematic Review of Human–Robot Interaction: The Use of Emotions and the Evaluation of Their Performance

A Systematic Review of Human–Robot Interaction: The Use of Emotions and the Evaluation of Their Performance

Variable Admittance Control of High Compatibility Exoskeleton Based on Human–Robotic Interaction Force

The wearable exoskeleton system is a typical strongly coupled human–robotic system. Human–robotic is the environment for each other. The two support each other and compete with each other. Achieving high human–robotic compatibility is the most critical technology for wearable systems. Full structural compatibility can improve the intrinsic safety of the exoskeleton, and precise intention understanding and motion control can improve the comfort of the exoskeleton. This paper first designs a physiologically functional bionic lower limb exoskeleton based on the study of bone and joint functional anatomy and analyzes the drive mapping model of the dual closed-loop four-link knee joint. Secondly, an exoskeleton dual closed-loop controller composed of a position inner loop and a force outer loop is designed. The inner loop of the controller adopts the PID control algorithm, and the outer loop adopts the adaptive admittance control algorithm based on human–robot interaction force (HRI). The controller can adaptively adjust the admittance parameters according to the HRI to respond to dynamic changes in the mechanical and physical parameters of the human–robot system, thereby improving control compliance and the wearing comfort of the exoskeleton system. Finally, we built a joint simulation experiment platform based on SolidWorks/Simulink to conduct virtual prototype simulation experiments and recruited volunteers to wear rehabilitation exoskeletons to conduct related control experiments. Experimental results show that the designed physiologically functional bionic exoskeleton and adaptive admittance controller can significantly improve the accuracy of human–robotic joint motion tracking, effectively reducing human–machine interaction forces and improving the comfort and safety of the wearer. This paper proposes a dual-closed loop four-link knee joint exoskeleton and a variable admittance control method based on HRI, which provides a new method for the design and control of exoskeletons with high compatibility.

Human-Centered Coordination for Robot-Assisted Equipment Transport

Abstract This work explores how to use an unmanned ground vehicle (UGV) to offload the physical burdens of equipment from humans. This work formulates Dynamic Alignment Following and compares it to Position-Based Following techniques. We describe the control strategies of both following methods and implement them in a dynamic simulation and a physical prototype. We test the performance of the two following methods and show that Dynamic Alignment Following can reduce robot positional error and interaction force between the human and the robot. We then analyze the energetics and the performance of the human-UGV team for candidate transportation tasks. The presence of the robot can make some tasks take longer to perform. Nonetheless, the results show that for the candidate tasks, the robot can reduce human average metabolic power and average overall task energy.

Using 3D Hand Pose Data in Recognizing Human–Object Interaction and User Identification for Extended Reality Systems

Object detection and action/gesture recognition have become imperative in security and surveillance fields, finding extensive applications in everyday life. Advancement in such technologies will help in furthering cybersecurity and extended reality systems through the accurate identification of users and their interactions, which plays a pivotal role in the security management of an entity and providing an immersive experience. Essentially, it enables the identification of human–object interaction to track actions and behaviors along with user identification. Yet, it is performed by traditional camera-based methods with high difficulties and challenges since occlusion, different camera viewpoints, and background noise lead to significant appearance variation. Deep learning techniques also demand large and labeled datasets and a large amount of computational power. In this paper, a novel approach to the recognition of human–object interactions and the identification of interacting users is proposed, based on three-dimensional hand pose data from an egocentric camera view. A multistage approach that integrates object detection with interaction recognition and user identification using the data from hand joints and vertices is proposed. Our approach uses a statistical attribute-based model for feature extraction and representation. The proposed technique is tested on the HOI4D dataset using the XGBoost classifier, achieving an average F1-score of 81% for human–object interaction and an average F1-score of 80% for user identification, hence proving to be effective. This technique is mostly targeted for extended reality systems, as proper interaction recognition and users identification are the keys to keeping systems secure and personalized. Its relevance extends into cybersecurity, augmented reality, virtual reality, and human–robot interactions, offering a potent solution for security enhancement along with enhancing interactivity in such systems.

High-Transparency Linear Actuator Using an Electromagnetic Brake for Damping Modulation in Physical Human–Robot Interaction

High-Transparency Linear Actuator Using an Electromagnetic Brake for Damping Modulation in Physical Human–Robot Interaction

Human–Robot interaction research in hospitality and tourism: trends and future directions

Purpose This paper aims to synthesize the evolving research of human–robot interaction (HRI) in the hospitality and tourism industry, identifying gaps and setting directions for future research. Design/methodology/approach Using a mixed-method approach, the study combines inductive co-citation analysis with deductive theory-context-characteristics-methodology analysis. Findings The findings trace the progression of HRI knowledge from initial feasibility and acceptance studies to advanced post-adoption experience management. The analysis identifies prevalent theories such as anthropomorphism theory, specific contexts like hotel environments, diverse robot types (e.g. embodied robots), outcome measures (e.g. use intention) and methodologies predominantly comprising survey-based analyses and experimental approaches. The analysis not only illuminates areas of research attention but also uncovers under-explored topics, offering a roadmap for future inquiry in tourism and HRI research. Originality/value The paper contributes to the literature by providing a structured framework that not only maps the intellectual structure of HRI research in tourism but also proposes a cohesive integration of disparate theories and methodologies, addressing both practical and academic gaps.