Robot Interaction Research Articles

Guessing Human Intentions to Avoid Dangerous Situations in Caregiving Robots

The integration of robots into social environments necessitates their ability to interpret human intentions and anticipate potential outcomes accurately. This capability is particularly crucial for social robots designed for human care, as they may encounter situations that pose significant risks to individuals, such as undetected obstacles in their path. These hazards must be identified and mitigated promptly to ensure human safety. This paper delves into the artificial theory of mind (ATM) approach to inferring and interpreting human intentions within human–robot interaction. We propose a novel algorithm that detects potentially hazardous situations for humans and selects appropriate robotic actions to eliminate these dangers in real time. Our methodology employs a simulation-based approach to ATM, incorporating a “like-me” policy to assign intentions and actions to human subjects. This strategy enables the robot to detect risks and act with a high success rate, even under time-constrained circumstances. The algorithm was seamlessly integrated into an existing robotics cognitive architecture, enhancing its social interaction and risk mitigation capabilities. To evaluate the robustness, precision, and real-time responsiveness of our implementation, we conducted a series of three experiments: (i) A fully simulated scenario to assess the algorithm’s performance in a controlled environment; (ii) A human-in-the-loop hybrid configuration to test the system’s adaptability to real-time human input; and (iii) A real-world scenario to validate the algorithm’s effectiveness in practical applications. These experiments provided comprehensive insights into the algorithm’s performance across various conditions, demonstrating its potential for improving the safety and efficacy of social robots in human care settings. Our findings contribute to the growing research on social robotics and artificial intelligence, offering a promising approach to enhancing human–robot interaction in potentially hazardous environments. Future work may explore the scalability of this algorithm to more complex scenarios and its integration with other advanced robotic systems.

Robot Control Platform for Multimodal Interactions with Humans Based on ChatGPT

This paper presents the architecture of a multimodal human–robot interaction control platform that leverages the advanced language capabilities of ChatGPT to facilitate more natural and engaging conversations between humans and robots. Implemented on the Pepper humanoid robot, the platform aims to enhance communication by providing a richer and more intuitive interface. The motivation behind this study is to enhance robot performance in human interaction through cutting-edge natural language processing technology, thereby improving public attitudes toward robots, fostering the development and application of robotic technology, and reducing the negative attitudes often associated with human–robot interactions. To validate the system, we conducted experiments measuring negative attitude robot scale and their robot anxiety scale scores before and after interacting with the robot. Statistical analysis of the data revealed a significant improvement in the participants’ attitudes and a notable reduction in anxiety following the interaction, indicating that the system holds promise for fostering more positive human–robot relationships.

Human–exoskeleton interaction portrait

Human–robot physical interaction contains crucial information for optimizing user experience, enhancing robot performance, and objectively assessing user adaptation. This study introduces a new method to evaluate human–robot interaction and co-adaptation in lower limb exoskeletons by analyzing muscle activity and interaction torque as a two-dimensional random variable. We introduce the interaction portrait (IP), which visualizes this variable’s distribution in polar coordinates. We applied IP to compare a recently developed hybrid torque controller (HTC) based on kinematic state feedback and a novel adaptive model-based torque controller (AMTC) with online learning, proposed herein, against a time-based controller (TBC) during treadmill walking at varying speeds. Compared to TBC, both HTC and AMTC significantly lower users’ normalized oxygen uptake, suggesting enhanced user-exoskeleton coordination. IP analysis reveals that this improvement stems from two distinct co-adaptation strategies, unidentifiable by traditional muscle activity or interaction torque analyses alone. HTC encourages users to yield control to the exoskeleton, decreasing overall muscular effort but increasing interaction torque, as the exoskeleton compensates for user dynamics. Conversely, AMTC promotes user engagement through increased muscular effort and reduces interaction torques, aligning it more closely with rehabilitation and gait training applications. IP phase evolution provides insight into each user’s interaction strategy formation, showcasing IP analysis’s potential in comparing and designing novel controllers to optimize human–robot interaction in wearable robots.

Active constraint control for the surgical robotic platform with concentric connector joints

Robotic minimally invasive surgery (MIS) has changed numerous surgical techniques in the past few years and enhanced their results. Haptic feedback is integrated into robotic surgical systems to restore the surgeon's perception of forces in response to interaction with objects in the surgical environment. The ideal exact emulation of the robot's interaction with its physical environment in free space is a very challenging problem to solve completely. Previously, we introduced the surgical robotic platform (SRP) with a novel concentric connector joint (CCJ). This study aims to develop a haptic control system that integrates an active constraint controller into a surgical robot platform. We have successfully established haptic feedback control for the surgical robot using constraint control and inverse kinematic relationships integrated into the overall positioning structure. A preliminary feasibility study, modelling, and simulation were presented.

Electronic Skin as Human–Robot Interface for Human Grasp Recognition in Home-Care Robots: Introducing a Complete Set of Resources

Population aging has highlighted the significance of developing home-care robots to assist human life. However, traditional home-care robots work according to fixed procedures, which lack human–robot interaction to mimic human hand action. Human–robot interfaces provide an opportunity for interactions between robots and users, which play an important role in home-care robots. Traditional human–robot interfaces are based on rigid, cumbersome, and high-priced machines, which prevent their wide application in performing home-care tasks. An e-skin sensor with 11 stretchable resistors evenly distributed on the back of the human hand, which could be applied to monitor human finger motion and control robotic fingers to dexterously grasp diverse objects, is presented in this article. A 3D printer is employed to pattern high-performance wires into structurally soft and stretchable filamentary serpentines. The stretchable resistors are fabricated by a thin sensing film of mixed multiwalled carbon nanotubes (MWCNTs) and silicone elastomer. The e-skin sensor is applied in a hand-in-the-loop system for controlling a robotic hand to help grasp an object synchronously. A robot system is developed to connect the e-skin with the robotic hand to perform grasp actions with a success rate of more than 80%. By analyzing the experimental results of the e-skin sensor and robotic hand grasp, this work obtains insights showing that the skin sensor can help engineers design a home-care assistant plan for patients according to monitoring value or a human–robot interaction mode by robotic fingers mimicking human finger actions.

Multipoint Variable parameter compliant control of redundant manipulator based on the equivalent twin model of flexible contact dynamics

To elucidate the dynamic coupling mechanism involved in multipoint/arbitrary point contact during human–robot interactions with redundant manipulators, we introduce a compliant control strategy that adapts to variable parameters. This strategy is based on the flexible contact dynamic equivalent twin system for human–robot interactions. Using the Virtual Elastic Element Representation model for arbitrary point contact dynamics and leveraging the skeleton principle for contact force conversion, we construct a flexible contact dynamic equivalent twin system tailored for redundant manipulators with multipoint/arbitrary point contact capabilities. Within this system, we implement real-time adjustments to the impedance stiffness coefficient in conjunction with the virtual end velocity of the manipulator. We optimize the joint angular velocity of the manipulator while considering the motion allocation of redundant degrees of freedom. This optimization enables multipoint contact variable parameter compliance control for safe human–robot interactions. To validate our approach, we analyze the multipoint contact dynamic model of the redundant manipulator's interaction with humans and assess the feasibility of the variable parameter compliance control method using robot operating system simulations. Ultimately, we verify the effectiveness and practicality of our proposed compliance control method through simulation and experimental results.

Safe physical human–robot interaction through variable impedance control based on ISO/TS 15066

Safe physical human–robot interaction through variable impedance control based on ISO/TS 15066

Open-set 3D semantic instance maps for vision language navigation – O3D-SIM

Humans excel at forming mental maps of their surroundings, equipping them to understand object relationships and navigate based on language queries. Our previous work SI Maps (Nanwani L, Agarwal A, Jain K, et al. Instance-level semantic maps for vision language navigation. In: 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). IEEE; 2023 Aug.) showed that having instance-level information and the semantic understanding of an environment helps significantly improve performance for language-guided tasks. We extend this instance-level approach to 3D while increasing the pipeline's robustness and improving quantitative and qualitative results. Our method leverages foundational models for object recognition, image segmentation, and feature extraction. We propose a representation that results in a 3D point cloud map with instance-level embeddings, which bring in the semantic understanding that natural language commands can query. Quantitatively, the work improves upon the success rate of language-guided tasks. At the same time, we qualitatively observe the ability to identify instances more clearly and leverage the foundational models and language and image-aligned embeddings to identify objects that, otherwise, a closed-set approach wouldn't be able to identify. Project Page - https://smart-wheelchair-rrc.github.io/o3d-sim-webpage

Human-Social Robot Interaction in the Light of ToM and Metacognitive Functions

Theory of Mind (ToM) and Metacognition constitute two superior mental mechanisms that promote the smooth integration and adaptation of the individual in society. In particular, the ability to read minds introduces the individual into the social world, contributing to understanding oneself and others. Metacognition focuses on individual knowledge, control, regulation, and readjustment regarding the cognitive mechanism and its influence on cognitive performance and the mental and social development of the individual. At the basis of the development of the two mechanisms is the activation of social interaction, which determines their levels of development. The innovative approaches and great expectations of technology and Artificial Intelligence for improving the artificial mind brought social robots to the fore. Robots with social action are gradually entering human life. Their interaction with the human factor is anticipated to become more and more frequent, expanded, and specialized. Hence, the investigation of equipping artificial systems with integrated social-cognitive and metacognitive capabilities was necessary, constituting the subject of study of the current narrative review. Research findings show that intelligent systems with introspection, self-evaluation, and perception-understanding of emotions, intentions, and beliefs can develop safe and satisfactory communication with humans as long as their design and operation conform to the code of ethics.

Anthropomorphism-based artificial intelligence (AI) robots typology in hospitality and tourism

Purpose Anthropomorphism plays a crucial role in the deployment of human-like robots in hospitality and tourism. This study aims to propose an anthropomorphism-based typology of artificial intelligence (AI) robots, based on robot attributes, usage, function and application across different operational levels. Design/methodology/approach Following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) checklist, the research was conducted in two stages. A search strategy was implemented to explore anthropomorphism-based AI robots and to develop a robot typology. Findings This study provides a comprehensive typology of anthropomorphism-based AI robots used in tourism and hospitality and classifies them into four types, namely, chatbots, mechanoids, humanoids and android robots. Each type features distinct functions and applications. Practical implications The findings can assist companies in using anthropomorphic robots to improve service and strengthen competitiveness. This study offers valuable insights to managers for deploying AI robots across diverse service sectors. Originality/value This research provides a novel typology of hospitality and tourism AI robots and extends the understanding of anthropomorphism in human–robot interaction. This typology encompasses both virtual and physical robots, providing clarity on their attributes, usage, functions and applications across diverse areas of hospitality operations.

Grasp and remember: the impact of human and robotic actions on object preference and memory

Goal contagion, the tendency to adopt others' goals, significantly impacts cognitive processes, which gains particular importance in the emerging field of human–robot interactions. The present study explored how observing human versus robotic actions affects preference and memory. Series of objects undergoing either human or robotic grasping actions together with static (no action) objects were presented, while participants indicated their preference for each object. After a short delay, their memory for grasped, static and new (unstudied) stimuli was tested. Human actions enhanced preference and subsequent recollection of objects, more than robotic actions. In the context of human action, static objects were also perceived as more familiar at recognition. The goal contagion's influence on memory was found to be independent from its impact on preference. These findings highlight the critical role of human interaction in eliciting the impact of goal contagion on cognitive evaluations, memory engagement and the creation of detailed associative memories.

Correction to: A human–robot interaction control strategy for teleoperation robot system under multi‑scenario applications

Correction to: A human–robot interaction control strategy for teleoperation robot system under multi‑scenario applications

Sustainable Impact of Stance Attribution Design Cues for Robots on Human–Robot Relationships—Evidence from the ERSP

With the development of large language model technologies, the capability of social robots to interact emotionally with users has been steadily increasing. However, the existing research insufficiently examines the influence of robot stance attribution design cues on the construction of users’ mental models and their effects on human–robot interaction (HRI). This study innovatively combines mental models with the associative–propositional evaluation (APE) model, unveiling the impact of the stance attribution explanations of this design cue on the construction of user mental models and the interaction between the two types of mental models through EEG experiments and survey investigations. The results found that under the influence of intentional stance explanations (compared to design stance explanations), participants displayed higher error rates, higher θ- and β-band Event-Related Spectral Perturbations (ERSPs), and phase-locking value (PLV). Intentional stance explanations trigger a primarily associatively based mental model of users towards robots, which conflicts with the propositionally based mental models of individuals. Users might adjust or “correct” their immediate reactions caused by stance attribution explanations after logical analysis. This study reveals that stance attribution interpretation can significantly affect users’ mental model construction of robots, which provides a new theoretical framework for exploring human interaction with non-human agents and provides theoretical support for the sustainable development of human–robot relations. It also provides new ideas for designing robots that are more humane and can better interact with human users.

Application of gamification based virtual robots in urban landscape Design: Interaction and entertainment experience in the design process

The traditional design process lacks fun and participation, so new elements need to be introduced to enhance the attraction and creativity of the design. The goal of the research is to design an urban landscape design method based on gamified elements and virtual robots to increase the interactivity and entertainment experience in the design process, and to explore its impact on the design results. This paper proposes an urban landscape design framework based on gamified elements and virtual robots, which includes multiple stages in the design process, each of which introduces different gamified tasks and interactions of virtual robots. Designers can gain new design ideas and inspiration by completing tasks and interacting with virtual robots. The study evaluated the effects of design methods using gamification elements and virtual robots on the design process and design results. The results show that this approach effectively increases the engagement and enjoyment of the design process, while also promoting innovation and sustainability of the design results. The urban landscape design method based on gamification elements and virtual robots has broad application prospects, which can bring more creativity and fun to urban landscape design.

Human-redundant robot interaction and redundancy utilization based on three-dimensional force sensor

To make full use of redundant characteristics, the inverse kinematics algorithm based on augmented Jacobian matrix with the goal of avoiding joint limit and obstacle is adopted. Then, admittance control system is established for human-robot interaction, and force control is converted into motion control of robot. Through the expansion of Jacobian matrix, the model of joint limit avoidance is established. Regarding obstacle avoidance in human-robot interaction process, the potential function gradient is calculated and augmented Jacobian matrix is formed. Thus, comprehensive control model of human-robot interaction and obstacle avoidance is established. When the external force is applied, the robot can move smoothly along the human hand. The robot's motion process is smooth and compliant. At the same time, when performing task of avoiding joint limit, the values of the joint angle are within the set range. The average angle margin of the four joints is over 60 %. So the goal of avoiding the joint limit is achieved. When adding obstacle avoidance in the interaction control, the difference value between the minimum distance and safe distance is always positive. The safety margin of the distance is over 20 %. So the robot does not contact with the obstacle. So human-robot interaction and redundancy utilization can be realized simultaneously.

Study of Human–Robot Interactions for Assistive Robots Using Machine Learning and Sensor Fusion Technologies

In recent decades, the potential of robots’ understanding, perception, learning, and action has been widely expanded due to the integration of artificial intelligence (AI) into almost every system. Cooperation between AI and human beings will be responsible for the bright future of AI technology. Moreover, for a perfect manually or automatically controlled machine or device, the device must perform together with a human through multiple levels of automation and assistance. Humans and robots cooperate or interact in various ways. With the enhancement of robot efficiencies, they can perform more work through an automatic method; therefore, we need to think about cooperation between humans and robots, the required software architectures, and information about the designs of user interfaces. This paper describes the most important strategies of human–robot interactions and the relationships between several control techniques and cooperation techniques using sensor fusion and machine learning (ML). Based on the behavior and thinking of humans, a human–robot interaction (HRI) framework is studied and explored in this article to make attractive, safe, and efficient systems. Additionally, research on intention recognition, compliance control, and perception of the environment by elderly assistive robots for the optimization of HRI is investigated in this paper. Furthermore, we describe the theory of HRI and explain the different kinds of interactions and required details for both humans and robots to perform different kinds of interactions, including the circumstances-based evaluation technique, which is the most important criterion for assistive robots.

A soft robotic system imitating the multimodal sensory mechanism of human fingers for intelligent grasping and recognition

The sensory function is crucial for achieving precise feedback control and environmental interaction in soft robots. Therefore, the multimodal sensory capabilities that mimic human fingers have always been a research hotspot. This study proposes a design method for a soft robotic gripper that can emulate the multimodal perception mechanism of human fingers and achieve high-precision recognition of grasped objects using machine learning algorithms. In addition, a finger texture structure inspired by human fingerprints is designed using a negative pressure-induced buckling method to enhance the soft gripper's ability to perceive minute surface textures. Inspired by slow adapting (SA) receptors and fast adapting (FA) receptors of human fingers, we have innovatively designed a capacitive curvature sensor (CS) and a triboelectric texture sensor (TS) to detect the size, material, and texture of objects. Furthermore, a non-contact medium identification sensor (MIS), having a sensitivity of more than 1300 times higher compared to the traditional MISs, is fabricated using the principle of electromagnetic resonance. It is shown that by employing a deep learning-based multi-channel feature fusion network, the soft grasping system with multimodal sensing has achieved a recognition accuracy of 98.43 % for different objects. This work provides an innovative approach for the application of soft robots in various fields, such as logistics sorting and food safety.

Interactive robots for personalised multimodal comedy experiments

This research proposes a novel system for personalising humour delivery using interactive robots with AI, natural language processing, and human-robot interaction technologies. The robots can adapt the multimodal content of jokes tactically now and strategically over time based on recording and analyzing each user’s subjective emotional reactions. The study evaluated pragmatic aspects of timing and framing jokes by having the robots try different “comeback tactics” and observing audience response. An ANOVA analysis revealed six distinct response categories. 78 % of participants positively rated the personalized robot comedy across the six scenarios. However, when both robot comedians took a negative tone, only 23 % approved (p < 0.001). Interestingly, audiences validated the robot comedian’s performance (89 % approval) more than the human comedian (54 % approval), a statistically significant difference (p < 0.01). These findings shed light on ideal characteristics for effective human-robot comedy interactions, highlighting the importance of topicality, maintaining a generally positive mood, and the unique strengths of robot performers. The study paves the way for further optimization of personalized, adaptive robot humour systems leveraging multimodal AI capabilities to enhance entertainment experiences tailored to individuals.

A Meta-Analysis of Vulnerability and Trust in Human–Robot Interaction

In human–robot interaction studies, trust is often defined as a process whereby a trustor makes themselves vulnerable to a trustee. The role of vulnerability however is often overlooked in this process but could play an important role in the gaining and maintenance of trust between users and robots. To better understand how vulnerability affects human–robot trust, we first reviewed the literature to create a conceptual model of vulnerability with four vulnerability categories. We then performed a meta-analysis, first to check the overall contribution of the variables included on trust. The results showed that overall, the variables investigated in our sample of studies have a positive impact on trust. We then conducted two multilevel moderator analysis to assess the effect of vulnerability on trust, including: (1) an intercept model that considers the relationship between our vulnerability categories and (2) a non-intercept model that treats each vulnerability category as an independent predictor. Only model 2 was significant, suggesting that to build trust effectively, research should focus on improving robot performance in situations where the users are unsure how reliable the robot will be. As our vulnerability variable is derived from studies of human–robot interaction and researcher reflections about the different risks involved, we relate our findings to these domains and make suggestions for future research avenues.

SUPER-MAN: SUPERnumerary robotic bodies for physical assistance in huMAN–robot conjoined actions

This paper presents a mobile supernumerary robotic approach to physical assistance in human–robot conjoined actions. The study starts with the description of the SUPER-MAN concept. The idea is to develop and utilize mobile collaborative systems that can follow human loco-manipulation commands to perform industrial tasks through three main components: (i) an admittance-type interface, (ii) a human–robot interaction controller and (iii) a supernumerary robotic body. Next, we present two possible implementations within the framework — from theoretical and hardware perspectives. The first system is called MOCA-MAN, and is composed of a redundant torque-controlled robotic arm and an omni-directional mobile platform. The second one is called Kairos-MAN, formed by a high-payload 6-DoF velocity-controlled robotic arm and an omni-directional mobile platform. The systems share the same admittance interface, through which user wrenches are translated to loco-manipulation commands, generated by whole-body controllers of each system. Besides, a thorough user-study with multiple and cross-gender subjects is presented to reveal the quantitative performance of the two systems in effort demanding and dexterous tasks. Moreover, we provide qualitative results from the NASA-TLX questionnaire to demonstrate the SUPER-MAN approach’s potential and its acceptability from the users’ viewpoint.