Human-robot Interaction Model Research Articles

Human-redundant robot interaction and redundancy utilization based on three-dimensional force sensor

To make full use of redundant characteristics, the inverse kinematics algorithm based on augmented Jacobian matrix with the goal of avoiding joint limit and obstacle is adopted. Then, admittance control system is established for human-robot interaction, and force control is converted into motion control of robot. Through the expansion of Jacobian matrix, the model of joint limit avoidance is established. Regarding obstacle avoidance in human-robot interaction process, the potential function gradient is calculated and augmented Jacobian matrix is formed. Thus, comprehensive control model of human-robot interaction and obstacle avoidance is established. When the external force is applied, the robot can move smoothly along the human hand. The robot's motion process is smooth and compliant. At the same time, when performing task of avoiding joint limit, the values of the joint angle are within the set range. The average angle margin of the four joints is over 60 %. So the goal of avoiding the joint limit is achieved. When adding obstacle avoidance in the interaction control, the difference value between the minimum distance and safe distance is always positive. The safety margin of the distance is over 20 %. So the robot does not contact with the obstacle. So human-robot interaction and redundancy utilization can be realized simultaneously.

Implementation of Engagement Detection for Human-Robot Interaction in Complex Environments.

This study develops a comprehensive robotic system, termed the robot cognitive system, for complex environments, integrating three models: the engagement model, the intention model, and the human-robot interaction (HRI) model. The system aims to enhance the naturalness and comfort of HRI by enabling robots to detect human behaviors, intentions, and emotions accurately. A novel dual-arm-hand mobile robot, Mobi, was designed to demonstrate the system's efficacy. The engagement model utilizes eye gaze, head pose, and action recognition to determine the suitable moment for interaction initiation, addressing potential eye contact anxiety. The intention model employs sentiment analysis and emotion classification to infer the interactor's intentions. The HRI model, integrated with Google Dialogflow, facilitates appropriate robot responses based on user feedback. The system's performance was validated in a retail environment scenario, demonstrating its potential to improve the user experience in HRIs.

A unified beamforming and source separation model for static and dynamic human-robot interaction.

This paper presents a unified model for combining beamforming and blind source separation (BSS). The validity of the model's assumptions is confirmed by recovering target speech information in noise accurately using Oracle information. Using real static human-robot interaction (HRI) data, the proposed combination of BSS with the minimum-variance distortionless response beamformer provides a greater signal-to-noise ratio (SNR) than previous parallel and cascade systems that combine BSS and beamforming. In the difficult-to-model HRI dynamic environment, the system provides a SNR gain that was 2.8 dB greater than the results obtained with the cascade combination, where the parallel combination is infeasible.

Task Space Compliant Control and Six-Dimensional Force Regulation Toward Automated Robotic Ultrasound Imaging

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Objective:</i> We propose a general control framework for task space compliant motion and six-dimensional (6-D) force regulation towards automated robotic ultrasound (US) imaging. The framework endows a position-controlled robotic manipulator with the capability of accurate compliant motion in free space and accurate force control in motion-constrained environment. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Methods:</i> An intuitive six degree-of-freedom (6-DoF) admittance control model expressed in an arbitrary Cartesian body frame is mathematically derived with closed-form task space error mapping. Its practical implementation on widely-used collaborative manipulators is proposed to achieve full task space compliant behaviors and accurate 6-D force control. A hybrid control law is presented to achieve good motion accuracy in free space and improved coupled stability in motion-constrained environment. The coupled model of physical human-robot interaction is established and the reason for the improved coupled stability is analyzed through simulation. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Results:</i> Evaluation experiments on the proposed control framework were performed to show the effectiveness. The mean error of compliant trajectory following was less than 0.30 mm in free space. The mean relative force and moment control accuracy in three orthogonal directions was better than 0.5% and 0.8%, respectively. The improved coupled stability under the same model parameters was also confirmed by human-robot interaction experiments. Finally, an automated robotic US imaging experiment on a human volunteer in a real clinical scenario was carried out to show the potential application of our proposed framework. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Conclusion:</i> Experimental results have shown the advantages of the control framework, including satisfied force control accuracy, high accuracy of compliant motion, improved coupled stability, and system effectiveness on a human volunteer. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper was motivated by the increasing needs of automated ultrasound (US) scanning for both diagnostic and interventional purpose. Clinical sonographers suffer from repeated workload when performing diagnostic US imaging, which could benefit from automated robotic scanning. Robotic US imaging involves physical interaction between the robot end-effector (i.e., US transducer) and the human body. The dynamics of the interaction is regulated by the control law to guarantee the contact of the US transducer and the safety of the procedure. Most existing works have focused on regulating in-plane contact force in terms of the position without considering the compliance in other dimensions. However, it is not a trivial work to extend the positional compliance to six degree-of-freedom (6-DoF) compliance. As the prevalence of low-cost collaborative robotic arms in medical scenarios, how to perform 6-DoF compliant trajectory following and accurate six-dimensional (6-D) force control on these robotic arms becomes increasingly important. This paper gives a complete general solution to achieve 6-DoF compliant control and 6-D force regulation with accurate kinematics on a position-controlled robotic arm. A hybrid control law is proposed to switch the government of “instantaneous model” and “theoretical model” to achieve compliant motion accuracy in free space and improved coupled stability in motion-constrained environment. No expensive torque sensors and torque control interface are required. And no prior geometric knowledge about the scanning object is needed. We have demonstrated the application for robotic US imaging in a real clinical scenario.

The Social Triad Model: Considering the Deployer in a Novel Approach to Trust in Human–Robot Interaction

There is an increasing interest in considering, measuring, and implementing trust in human-robot interaction (HRI). New avenues in this field include identifying social means for robots to influence trust, and identifying social aspects of trust such as a perceptions of robots’ integrity, sincerity or even benevolence. However, questions remain regarding robots’ authenticity in obtaining trust through social means and their capacity to increase such experiences through social interaction with users. We propose that the dyadic model of HRI misses a key complexity: a robot’s trustworthiness may be contingent on the user’s relationship with, and opinion of, the individual or organisation deploying the robot (termed here, Deployer). We present a case study in three parts on researching HRI and a LEGO® Serious® Play focus group on care robotics to indicate how Users’ trust towards the Deployer can affect trust towards robots and robotic research. Our Social Triad model (User, Robot, Deployer) offers novel avenues for exploring trust in a social context.

Agent talks about itself: an implementation using Jason, CArtAgO and Speech Acts

Thinking to oneself is a prerogative of man when he needs to think about or repeat what he is doing or experiencing. It is a way of processing information and setting in motion a decision-making process. When this is done aloud, there is also a chance that someone else will understand the meaning or reasons for the action. Equipping an agent with the ability to reveal the reasons for its decisions is both a way to improve human interaction and a way to improve the triggering of a decision process. In this work, we propose to use the speech act to enable a coalition of agents to exhibit inner speech capabilities to explain their behavior, but also to guide and reinforce the creation of an inner model. The BDI agent paradigm, Jason, and CArtAgO are used to give agents the ability to act in a human-like manner. The BDI reasoning cycle has been extended to include inner speech. The proposed solution continues the research path that started with the definition of a cognitive model and architecture for human-robot teaming interaction and aims to integrate the believable interaction paradigm in it.

Fuzzy-Based Optimization and Control of a Soft Exosuit for Compliant Robot–Human–Environment Interaction

Many previous studies of soft exosuits improved human locomotion performance. However, there is no example to control a soft exosuit using human ankle impedance adaption in assistance tasks compliantly. In this article, the human–environment interaction information is exploited into the exosuit control. A novel fuzzy-based optimization and control method of soft exosuit is proposed to provide plantarflexion assistance for human walking by changing the human–robot interaction. In particular, a fuzzy neurodynamics optimization is developed to learn the unknown human ankle impedance parameters automatically. A fuzzy approximation technique is applied to improve the control performance of the exosuit when a human is walking with unknown human–robot interaction model parameters. This control scheme guarantees that the human–robot dynamics follows a target human ankle impedance model to obtain the compliant interaction performance. Experiments on different participants verify the effectiveness of the control scheme. Results show that a compliant human–robot interaction is achieved by learning the human–environment interaction parameters, i.e., the human ankle parameters. It indicates that our proposed method can facilitate exosuit control to achieve compliant robot–human–environment interaction.

Adaptive-Constrained Impedance Control for Human-Robot Co-Transportation.

Human-robot co-transportation allows for a human and a robot to perform an object transportation task cooperatively on a shared environment. This range of applications raises a great number of theoretical and practical challenges arising mainly from the unknown human-robot interaction model as well as from the difficulty of accurately model the robot dynamics. In this article, an adaptive impedance controller for human-robot co-transportation is put forward in task space. Vision and force sensing are employed to obtain the human hand position, and to measure the interaction force between the human and the robot. Using the latest developments in nonlinear control theory, we propose a robot end-effector controller to track the motion of the human partner under actuators' input constraints, unknown initial conditions, and unknown robot dynamics. The proposed adaptive impedance control algorithm offers a safe interaction between the human and the robot and achieves a smooth control behavior along the different phases of the co-transportation task. Simulations and experiments are conducted to illustrate the performance of the proposed techniques in a co-transportation task.

Asymmetric Identification Model for Human-Robot Contacts via Supervised Learning

Human-robot interaction (HRI) occupies an essential role in the flourishing market for intelligent robots for a wide range of asymmetric personal and entertainment applications, ranging from assisting older people and the severely disabled to the entertainment robots at amusement parks. Improving the way humans and machines interact can help democratize robotics. With machine and deep learning techniques, robots will more easily adapt to new tasks, conditions, and environments. In this paper, we develop, implement, and evaluate the performance of the machine-learning-based HRI model in a collaborative environment. Specifically, we examine five supervised machine learning models viz. the ensemble of bagging trees (EBT) model, the k-nearest neighbor (kNN) model, the logistic regression kernel (LRK), the fine decision trees (FDT), and the subspace discriminator (SDC). The proposed models have been evaluated on an ample and modern contact detection dataset (CDD 2021). CDD 2021 is gathered from a real-world robot arm, Franka Emika Panda, when it was executing repetitive asymmetric movements. Typical performance assessment factors are applied to assess the model effectiveness in terms of detection accuracy, sensitivity, specificity, speed, and error ratios. Our experiential evaluation shows that the ensemble technique provides higher performance with a lower error ratio compared with other developed supervised models. Therefore, this paper proposes an ensemble-based bagged trees (EBT) detection model for classifying physical human–robot contact into three asymmetric types of contacts, including noncontact, incidental, and intentional. Our experimental results exhibit outstanding contact detection performance metrics scoring 97.1%, 96.9%, and 97.1% for detection accuracy, precision, and sensitivity, respectively. Besides, a low prediction overhead has been observed for the contact detection model, requiring a 102 µS to provide the correct detection state. Hence, the developed scheme can be efficiently adopted through the application requiring physical human–robot contact to give fast accurate detection to the contacts between the human arm and the robot arm.

Service humanoid robotics: a novel interactive system based on bionic-companionship framework.

At present, industrial robotics focuses more on motion control and vision, whereas humanoid service robotics (HSRs) are increasingly being investigated and researched in the field of speech interaction. The problem and quality of human-robot interaction (HRI) has become a widely debated topic in academia. Especially when HSRs are applied in the hospitality industry, some researchers believe that the current HRI model is not well adapted to the complex social environment. HSRs generally lack the ability to accurately recognize human intentions and understand social scenarios. This study proposes a novel interactive framework suitable for HSRs. The proposed framework is grounded on the novel integration of Trevarthen’s (2001) companionship theory and neural image captioning (NIC) generation algorithm. By integrating image-to-natural interactivity generation and communicating with the environment to better interact with the stakeholder, thereby changing from interaction to a bionic-companionship. Compared to previous research a novel interactive system is developed based on the bionic-companionship framework. The humanoid service robot was integrated with the system to conduct preliminary tests. The results show that the interactive system based on the bionic-companionship framework can help the service humanoid robot to effectively respond to changes in the interactive environment, for example give different responses to the same character in different scenes.

Human-centred adaptive control of lower limb rehabilitation robot based on human–robot interaction dynamic model

A rehabilitation robot can effectively alleviate the shortcomings and deficiencies of traditional treatments. The early stages of rehabilitation training use passive control based on a predefined trajectory. The goal of passive control is to control the motion of the robot, rather than the real motion of the human. However, the human and robot are connected by bands; thus, errors exist between them, and these errors affect the outcome of the rehabilitation training. This study proposes human-centred adaptive control of a lower limb rehabilitation robot based on a dynamic model of human–robot interactions. An equivalent spring model in three-dimensional space is proposed for expressing the interaction torque between the human and robot, reflecting the torque exerted by the robot on the human. A dynamic model for the human–robot system is established based on the human–robot interaction model. To overcome the uncertainty regarding the parameters of the human limbs and robots and that of the stiffness coefficient in the interaction force model, an adaptive controller is designed, and the stability of the system is analysed. The simulation results indicate the effectiveness of the controller.

Robotic Learning From Advisory and Adversarial Interactions Using a Soft Wrist

In this letter, we developed a novel learning framework from physical human-robot interactions. Owing to human domain knowledge, such interactions can be useful for facilitation of learning. However, applying numerous interactions for training data might place a burden on human users, particularly in real-world applications. To address this problem, we propose formulating this as a model-based reinforcement learning problem to reduce errors during training and increase robustness. Our key idea is to develop 1) an advisory and adversarial interaction strategy and 2) a human-robot interaction model to predict each behavior. In the advisory and adversarial interactions, a human guides and disturbs the robot when it moves in the wrong and correct directions, respectively. Meanwhile, the robot tries to achieve its goal in conjunction with predicting the human's behaviors using the interaction model. To verify the proposed method, we conducted peg-in-hole experiments in a simulation and real-robot environment with human participants and a robot, which has an underactuated soft wrist module. The experimental results showed that our proposed method had smaller position errors during training and a higher number of successes than the baselines without any interactions and with random interactions.

An Automated Planning Model for HRI: Use Cases on Social Assistive Robotics.

Using Automated Planning for the high level control of robotic architectures is becoming very popular thanks mainly to its capability to define the tasks to perform in a declarative way. However, classical planning tasks, even in its basic standard Planning Domain Definition Language (PDDL) format, are still very hard to formalize for non expert engineers when the use case to model is complex. Human Robot Interaction (HRI) is one of those complex environments. This manuscript describes the rationale followed to design a planning model able to control social autonomous robots interacting with humans. It is the result of the authors’ experience in modeling use cases for Social Assistive Robotics (SAR) in two areas related to healthcare: Comprehensive Geriatric Assessment (CGA) and non-contact rehabilitation therapies for patients with physical impairments. In this work a general definition of these two use cases in a unique planning domain is proposed, which favors the management and integration with the software robotic architecture, as well as the addition of new use cases. Results show that the model is able to capture all the relevant aspects of the Human-Robot interaction in those scenarios, allowing the robot to autonomously perform the tasks by using a standard planning-execution architecture.

MIHR: A Human-Robot Interaction Model

The interactions between people and social robots have generated positive effects on people of different ages in diverse contexts. A model of the interaction process is important to understand the person who interacts, in order to manage the internal dynamics of interaction in the social robots. There are models that describe the interaction between humans and machines, but they don't integrate the three most important elements to be considered during the interactions by the social robots: the modalities of human communication, the capacity of adaptation, and the expression of emotions. In this paper, a review of the interaction models between people and social robots is made, in order to analyze what has been done about these three important elements of the interaction. Then, it is proposed a Human-Robot Interaction Model (MIHR) based on a Human-Human Interaction Model (MIHH) previously developed, which integrates the main elements to be considered during the interactions by the social robots.

Real Time Crowd Navigation from First Principles of Probability Theory

Constructing realistic and real time human-robot interaction models is a core challenge in crowd navigation. In this paper we derive a robot-agent interaction density from first principles of probability theory; we call our approach “first order interacting Gaussian processes” (foIGP). Furthermore, we compute locally optimal solutions—with respect to multi-faceted agent “intent” and “flexibility”—in near real time on a laptop CPU. We test on challenging scenarios from the ETH crowd dataset and show that the safety and efficiency statistics of foIGP is competitive with human safety and efficiency statistics. Further, we compute the safety and efficiency statistics of dynamic window avoidance, a physics based model variant of foIGP, a Monte Carlo inference based approach, and the best performing deep reinforcement learning algorithm; foIGP outperforms all of them.

Quasi-Direct Drive Actuation for a Lightweight Hip Exoskeleton with High Backdrivability and High Bandwidth.

High-performance actuators are crucial to enable mechanical versatility of wearable robots, which are required to be lightweight, highly backdrivable, and with high bandwidth. State-of-the-art actuators, e.g., series elastic actuators (SEAs), have to compromise bandwidth to improve compliance (i.e., backdrivability). We describe the design and human-robot interaction modeling of a portable hip exoskeleton based on our custom quasi-direct drive (QDD) actuation (i.e., a high torque density motor with low ratio gear). We also present a model-based performance benchmark comparison of representative actuators in terms of torque capability, control bandwidth, backdrivability, and force tracking accuracy. This paper aims to corroborate the underlying philosophy of "design for control", namely meticulous robot design can simplify control algorithms while ensuring high performance. Following this idea, we create a lightweight bilateral hip exoskeleton to reduce joint loadings during normal activities, including walking and squatting. Experiments indicate that the exoskeleton is able to produce high nominal torque (17.5 Nm), high backdrivability (0.4 Nm backdrive torque), high bandwidth (62.4 Hz), and high control accuracy (1.09 Nm root mean square tracking error, 5.4% of the desired peak torque). Its controller is versatile to assist walking at different speeds and squatting. This work demonstrates performance improvement compared with state-of-the-art exoskeletons.

A Cloud-based Quadruped Service Robot with Multi-Scene Adaptability and various forms of Human-Robot Interaction

Abstract With the development of CPS technology, more robots are endowed with intelligence to interact and communicate with humans such as service robot AIBO and AMWELL. These intelligent pets show great human-machine interactive ability by voice and face recognition, storytelling. However, they are limited only in family scenes and fail to provide cross-scene services. Therefore, we design a quadruped service robot called Cloud Sphinx - whose motion, voice, and vision system are driven by artificial intelligent models in the cloud. Our work pushes the interaction between CPS and humans. First, an open-source 12-DoF robotic cat is rapidly prototyped with a 3D printing technique, and its versatile gestures and gaits are coordinated by advanced servo control algorithms. Second, by merging motion control, NLP, image processing deep learning and cloud computing, Cloud Sphinx can provide services as emotion classification, face recognition, and garbage classification through human-robot interaction models. Third, an RaaS (Robot as a service) architecture is developed to fulfill the incrementally learning from the human-robot interaction processes. Sufficient simulations and on-board tests verify the human-robot interaction functionality and cross-scene service performances. Applications that we designed show the A.I. performance of service robots can be effectively enhanced by model training during the human-robot interactions and knowledge accumulation in cloud database.

A Human Decision-Making Behavior Model for Human-Robot Interaction in Multi-Robot Systems

In this paper, a novel human decision-making behavior model is proposed for human-robot interaction (HRI) for the control of multi-robot systems (MRS). The proposed human drift diffusion model (HDDM) combines the traditional drift diffusion model (DDM) and the null-space-based behavioral control (NSBC) method by introducing a data-processing station and a human cognitive system. In the HDDM, the evolution of human-decision information is computed. By using a threshold of such information to trigger human interaction, accurate human decision-making timing can be obtained. In addition, a cooperative controller is designed for robots to follow human instructions. Simulations under various scenarios show that by using the proposed HDDM and the controller, robots can complete human instructions more accurately comparing to traditional methods. An experiment using a group of quadrotors subject to external wind disturbances also demonstrate the effectiveness of the proposed HDDM in real-world uncertain environments.

Human–robot interaction based on gesture and movement recognition

Human–robot interaction (HRI) has become a research hotspot in computer vision and robotics due to its wide application in human–computer interaction (HCI) domain. Based on the explored algorithms of gesture recognition and limb movement recognition in somatosensory interaction, an HRI model of a robotic arm is proposed for robot arm manipulation. More specifically, 3D SSD architecture is used for the location and identification of gesture and arm movement. Then, DTW template matching algorithm is adopted to trace the dynamic gestures. The interactive scenarios and interactive modes are designed for experiment and implementation. Virtual interactive experimental results have demonstrated the usefulness of our method.

Spine-Inspired Continuum Soft Exoskeleton for Stoop Lifting Assistance

Back injuries are the most prevalent work-related musculoskeletal disorders and represent a major cause of disability. Although innovations in wearable robots aim to alleviate this hazard, the majority of existing exoskeletons are obtrusive because the rigid linkage design limits natural movement, thus causing ergonomic risk. Moreover, these existing systems are typically only suitable for one type of movement assistance, not ubiquitous for a wide variety of activities. To fill in this gap, this letter presents a new wearable robot design approach continuum soft exoskeleton. This spine-inspired wearable robot is unobtrusive and assists both squat and stoops while not impeding walking motion. To tackle the challenge of the unique anatomy of spine that is inappropriate to be simplified as a single degree of freedom joint, our robot is conformal to human anatomy and it can reduce multiple types of forces along the human spine such as the spinae muscle force, shear, and compression force of the lumbar vertebrae. We derived kinematics and kinetics models of this mechanism and established an analytical biomechanics model of human-robot interaction. Quantitative analysis of disc compression force, disc shear force and muscle force was performed in simulation. We further developed a virtual impedance control strategy to deliver force control and compensate hysteresis of Bowden cable transmission. The feasibility of the prototype was experimentally tested on three healthy subjects. The root mean square error of force tracking is 6.63 N (3.3% of the 200 N peak force) and it demonstrated that it can actively control the stiffness to the desired value. This continuum soft exoskeleton represents a feasible solution with the potential to reduce back pain for multiple activities and multiple forces along the human spine.