Physical Human-robot Interaction Research Articles

Collaborative Bimanual Manipulation Using Optimal Motion Adaptation and Interaction Control: Retargeting Human Commands to Feasible Robot Control References

This article presents a robust and reliable human–robot collaboration (HRC) framework for bimanual manipulation. We propose an optimal motion adaptation method to retarget arbitrary human commands to feasible robot pose references while maintaining payload stability. The framework comprises three modules: 1) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a task-space sequential equilibrium and inverse kinematics optimization</i> ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">task-space SEIKO</i> ) for retargeting human commands and enforcing feasibility constraints, 2) an admittance controller to facilitate compliant human–robot physical interactions, and 3) a low-level controller improving stability during physical interactions. Experimental results show that the proposed framework successfully adapted infeasible and dangerous human commands into continuous motions within safe boundaries and achieved stable grasping and maneuvering of large and heavy objects on a real dual-arm robot via teleoperation and physical interaction. Furthermore, the framework demonstrated the capability in the assembly task of building blocks and the insertion task of industrial power connectors.

Design of an Assistive Controller for Physical Human–Robot Interaction Based on Cooperative Game Theory and Human Intention Estimation

Design of an Assistive Controller for Physical Human–Robot Interaction Based on Cooperative Game Theory and Human Intention Estimation

A Safety-Focused Admittance Control Approach for Physical Human–Robot Interaction With Rigid Multi-Arm Serial Link Exoskeletons

A Safety-Focused Admittance Control Approach for Physical Human–Robot Interaction With Rigid Multi-Arm Serial Link Exoskeletons

Vision-and Tactile-Based Continuous Multimodal Intention and Attention Recognition for Safer Physical Human–Robot Interaction

Employing skin-like tactile sensors on robots enhances both the safety and usability of collaborative robots by adding the capability to detect human contact. Unfortunately, simple binary tactile sensors alone cannot determine the context of the human contact—whether it is a deliberate interaction or an unintended collision that requires safety manoeuvres. Many published methods classify discrete interactions using more advanced tactile sensors or by analysing joint torques. Instead, we propose to augment the intention recognition capabilities of simple binary tactile sensors by adding a robot-mounted camera for human posture analysis. Different interaction characteristics, including touch location, human pose, and gaze direction, are used to train a supervised machine learning algorithm to classify whether a touch is intentional or not with an F1-score of 86%. We demonstrate that multimodal intention recognition is significantly more accurate than monomodal analyses with the collaborative robot Baxter. Furthermore, our method can also continuously monitor interactions that fluidly change between intentional or unintentional by gauging the user’s attention through gaze. If a user stops paying attention mid-task, the proposed intention and attention recognition algorithm can activate safety features to prevent unsafe interactions. We also employ a feature reduction technique that reduces the number of inputs to five to achieve a more generalized low-dimensional classifier. This simplification both reduces the amount of training data required and improves real-world classification accuracy. It also renders the method potentially agnostic to the robot and touch sensor architectures while achieving a high degree of task adaptability. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Whenever a user interacts physically with a robot, such as in collaborative manufacturing, the robot may respond to unintended touch inputs from the user. This may be through body collisions or that the user is suddenly distracted and is no longer paying attention to what they are doing. We propose an easy-to-implement method to augment safety of physical human-robot collaboration by determining whether the touch from a user is intentional or not through the use of robot-mounted basic touch sensors and computer vision. The algorithm examines the location of the user’s hands relative to the touched sensors in addition to observing where the user is looking. Machine learning is then used to classify in real-time, with an F1-score of 86%, whether a touch is intentional or not such that the robot can react accordingly. The method is particularly applicable in collaborative manufacturing contexts, but can also be applied anywhere where a user physically interacts with a robot. We demonstrate the utility of the method in enhancing safety during human-robot collaboration through a simulated collaborative manufacturing scenario with the robot Baxter, but the method can easily be adapted to other systems.

Physical Human–Robot Interaction Based on Adaptive Impedance Control for Robotic-Assisted Total Hip Arthroplasty

Physical Human–Robot Interaction Based on Adaptive Impedance Control for Robotic-Assisted Total Hip Arthroplasty

A Physical Human–Robot Interaction Framework for Trajectory Adaptation Based on Human Motion Prediction and Adaptive Impedance Control

A Physical Human–Robot Interaction Framework for Trajectory Adaptation Based on Human Motion Prediction and Adaptive Impedance Control

Task Space Compliant Control and Six-Dimensional Force Regulation Toward Automated Robotic Ultrasound Imaging

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Objective:</i> We propose a general control framework for task space compliant motion and six-dimensional (6-D) force regulation towards automated robotic ultrasound (US) imaging. The framework endows a position-controlled robotic manipulator with the capability of accurate compliant motion in free space and accurate force control in motion-constrained environment. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Methods:</i> An intuitive six degree-of-freedom (6-DoF) admittance control model expressed in an arbitrary Cartesian body frame is mathematically derived with closed-form task space error mapping. Its practical implementation on widely-used collaborative manipulators is proposed to achieve full task space compliant behaviors and accurate 6-D force control. A hybrid control law is presented to achieve good motion accuracy in free space and improved coupled stability in motion-constrained environment. The coupled model of physical human-robot interaction is established and the reason for the improved coupled stability is analyzed through simulation. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Results:</i> Evaluation experiments on the proposed control framework were performed to show the effectiveness. The mean error of compliant trajectory following was less than 0.30 mm in free space. The mean relative force and moment control accuracy in three orthogonal directions was better than 0.5% and 0.8%, respectively. The improved coupled stability under the same model parameters was also confirmed by human-robot interaction experiments. Finally, an automated robotic US imaging experiment on a human volunteer in a real clinical scenario was carried out to show the potential application of our proposed framework. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Conclusion:</i> Experimental results have shown the advantages of the control framework, including satisfied force control accuracy, high accuracy of compliant motion, improved coupled stability, and system effectiveness on a human volunteer. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper was motivated by the increasing needs of automated ultrasound (US) scanning for both diagnostic and interventional purpose. Clinical sonographers suffer from repeated workload when performing diagnostic US imaging, which could benefit from automated robotic scanning. Robotic US imaging involves physical interaction between the robot end-effector (i.e., US transducer) and the human body. The dynamics of the interaction is regulated by the control law to guarantee the contact of the US transducer and the safety of the procedure. Most existing works have focused on regulating in-plane contact force in terms of the position without considering the compliance in other dimensions. However, it is not a trivial work to extend the positional compliance to six degree-of-freedom (6-DoF) compliance. As the prevalence of low-cost collaborative robotic arms in medical scenarios, how to perform 6-DoF compliant trajectory following and accurate six-dimensional (6-D) force control on these robotic arms becomes increasingly important. This paper gives a complete general solution to achieve 6-DoF compliant control and 6-D force regulation with accurate kinematics on a position-controlled robotic arm. A hybrid control law is proposed to switch the government of “instantaneous model” and “theoretical model” to achieve compliant motion accuracy in free space and improved coupled stability in motion-constrained environment. No expensive torque sensors and torque control interface are required. And no prior geometric knowledge about the scanning object is needed. We have demonstrated the application for robotic US imaging in a real clinical scenario.

Considerations on the Dynamics of Biofidelic Sensors in the Assessment of Human–Robot Impacts

Ensuring the safety of physical human–robot interaction (pHRI) is of utmost importance for industries and organisations seeking to incorporate robots into their workspaces. To address this concern, the ISO/TS 15066:2016 outlines hazard analysis and preventive measures for ensuring safety in Human–Robot Collaboration (HRC). To analyse human–robot contact, it is common practice to separately evaluate the “transient” and “quasi-static” contact phases. Accurately measuring transient forces during close human–robot collaboration requires so-called “biofidelic” sensors that closely mimic human tissue properties, featuring adequate bandwidth and balanced damping. The dynamics of physical human–robot interactions using biofidelic measuring devices are being explored in this research. In this paper, one biofidelic sensor is tested to analyse its dynamic characteristics and identify the main factors influencing its performance and its practical applications for testing. To this aim, sensor parameters, such as natural frequency and damping coefficient, are estimated by utilising a custom physical pendulum setup to impact the sensor. Mathematical models developed to characterise the sensor system and pendulum dynamics are also disclosed.

Research on Force Interaction Devices Specialized for Blind Guidance Robot

Abstract: As a new type of human-computer interaction, physical human-robot interaction is very common in guide robots and is the mainstream of future development. However, the force interaction device is hard to choose according to actual application requirements. In this paper, we compared the force transmission characteristics of three interaction mediums: rigid rod, spring-damping, and cable. The result shows that the rigid rod will bring a force impact on blind users when the robot’s motion state switches, while the spring-damper can not perform well in its response and stability. In contrast, the cable is more suitable in this area for its fast response and stability, moreover, it does not impose a dramatic impact when imposing an excitation force.

A DMP-Based Motion Generation Scheme for Robotic Mirror Therapy

Hemiparesis has become prevalent in recent years, and robot-assisted rehabilitation provides great therapeutic potential. Robotic mirror therapy is a promising approach for hemiplegic patients by transmitting the motion of the healthy limb (HL) to the impaired limb (IL), where the wearable robot assists the IL to mimic the HL's action to stimulate the active participation of the injured muscles. However, complete replication of the HL's movement trajectory without considering the IL's muscle strength is not satisfactory for rehabilitation. In this article, a motion generation scheme is proposed for robotic mirror therapy. The robot's movement trajectory is modeled with dynamic movement primitive (DMP), and the physical human–robot interaction is formulated as an impedance model and coupled with the DMP model. The subject's muscle strength is evaluated with skin surface electromyography signals and transferred to the robot's joint stiffness. In order to adapt the human–robot coupled DMP model parameters to varying subject performance, a reinforcement learning algorithm is designed to optimize the robot's movement trajectory and stiffness profile, where the training safety and rehabilitation improvement are both guaranteed. The proposed method is validated using a lower extremity rehabilitation robot, and experimental results demonstrate its feasibility and superiority.

Physical Human–Robot Interaction Using a Macro–Mini Robotic System

In this article, we investigate the characteristics of a collaborative hybrid parallel robot. A comparison between this robot and other collaborative robots regarding the Cartesian inertia is presented to highlight the very low inertia of the hybrid parallel robot. A task space stiffness-damping control is also proposed. In order to improve the performance of macro--mini systems in the interaction context, an active macro-active mini system consisting of the hybrid parallel robot mounted on a three degrees of freedom (3-DOF) gantry translational robot is introduced. A control strategy for physical human–robot interaction is applied to the macro--mini arrangement. The stability of this method is analyzed and it is shown that the macro–mini combination is more stable than the mini alone. An experimental validation is then carried out. The results obtained with the hybrid parallel robot show that the desired interaction force can be tracked at high speed and that prescribed impedance parameters can be precisely rendered to the human operator without using any additional sensor. Also, it is pointed out that the bandwidth of interaction of the hybrid parallel robot, as well as of the macro--mini system is much higher than that of other commercial collaborative robots. In addition, other tests are realized on the macro–mini system in order to verify the performance and to demonstrate potential applications. The macro–mini robot introduced in this work yields a very intuitive human–robot interaction, which makes it ideal for many applications in which direct physical teaching or assistance is needed.

Variable admittance control for safe physical human–robot interaction considering intuitive human intention

The trajectory planning of the arm is greatly facilitated by the physical direct teaching technology. Variable admittance control is a promising technology when a robot is interacting with a variable environment, such as a human whose stiffness might change during the interaction. Nevertheless, a canonical admittance controller with imperfect parameters may lead to robot oscillation, which brings more challenges to a variable admittance controller. In this paper, we propose an energy-based variable admittance controller with intrinsic oscillation suppression property. During the physical human–robot interaction (pHRI), the human intention is predicted based on the robot’s state and interaction force. The admittance parameters are tuned automatically to conform the robot’s motion to human intention. When the oscillation is detected online by the proposed wavelet module, our variable admittance model reveals oscillation suppression ability because of dissipating the energy generated by high-frequency oscillation. We compared the proposed variable admittance controller with other admittance controller approaches in both simulation and actual robot experiments. The proposed method shows significant improvement in oscillation suppression and human energy conservation in the human–robot interaction application.

Modeling multi-contact point physical interaction between the anthropomorphic finger and soft robotic exo-digit for wearable rehabilitation robotics applications.

Introduction: Effective control of rehabilitation robots requires considering the distributed and multi-contact point physical human-robot interaction and users' biomechanical variation. This paper presents a quasi-static model for the motion of a soft robotic exo-digit while physically interacting with an anthropomorphic finger model for physical therapy. Methods: Quasi-static analytical models were developed for modeling the motion of the soft robot, the anthropomorphic finger, and their coupled physical interaction. An intertwining of kinematics and quasi-static motion was studied to model the distributed (multiple contact points) interaction between the robot and a human finger model. The anthropomorphic finger was modeled as an articulated multi-rigid body structure with multi-contact point interaction. The soft robot was modeled as an articulated hybrid soft-and-rigid model with a constant bending curvature and a constant length for each soft segment. A hyperelastic constitute model based on Yeoh's 3rdorder material model was used for modeling the soft elastomer. The developed models were experimentally evaluated for 1) free motion of individual soft actuators and 2) constrained motion of the soft robotic exo-digit and anthropomorphic finger model. Results and Discussion: Simulation and experimental results were compared for performance evaluations. The theoretical and experimental results were in agreement for free motion, and the deviation from the constrained motion was in the range of the experimental errors. The outcomes also provided an insight into the importance of considering lengthening for the soft actuators.

“It Brings the Good Vibes”: Exploring Biomorphic Aesthetics in the Design of Soft Personal Robots

The flexible bodies of soft robots provide exciting new possibilities for interaction with humans. In this paper, we propose a novel design paradigm, Soft Biomorphism, for soft robots centered on the idea of amplifying their inherent biomorphic aesthetic qualities and activating these as affordances for human interaction. Following this approach, we developed a set of biomorphic soft robotic prototypes and conducted two studies to understand the effects of biomorphic design aesthetics on people’s impressions of these prototypes. Based on qualitative data collected through five workshop sessions, the first exploratory study (n = 10) sought to investigate the envisioned uses and types of interactions that prototypes elicited within the context of personal robots. We found that various uses were considered and that most participants associated the biomorphic aesthetic design with soft robots contributing to emotional and physical well-being. Building on these results, we conducted a second study (n = 32) to investigate if soft robots with enhanced biomorphic qualities are perceived as more appealing and appropriate for physical human–robot interaction aimed at supporting well-being. We did not find any statistically significant preference for biomorphic soft robots. However, we found statistically significant differences in appeal ratings post-interaction for some prototypes, suggesting that physical interaction with soft robots can impact the perceived appeal. Based on our findings, we highlight key issues to bear in mind when considering biomorphic aesthetics in soft personal robot designs and provide tentative design recommendations to combine biomorphic and geometric elements and align visual appearance, tactility, and movement in future robot designs.

MyoPassivity Map: Does Multi-Channel sEMG Correlate With the Energetic Behavior of Upper-Limb Biomechanics During Physical Human-Robot Interaction?

MyoPassivity Map: Does Multi-Channel sEMG Correlate With the Energetic Behavior of Upper-Limb Biomechanics During Physical Human-Robot Interaction?

Towards Robust Physical Human Robot Interaction by an Adaptive Admittance Controller

Towards Robust Physical Human Robot Interaction by an Adaptive Admittance Controller

Adaptive Impedance Decentralized Control of Modular Robot Manipulators for Physical Human-robot Interaction

Adaptive Impedance Decentralized Control of Modular Robot Manipulators for Physical Human-robot Interaction

Variable Admittance Control in Sliding Mode for Robust Physical Human–Robot Interaction

Intuitive and comfortable physical human–robot interaction (pHRI) can be realized by changing impedance/admittance parameters corresponding to human interaction. However, this dynamic adjustment may result in drastically changed system dynamics, which usually give rise to system instability. We introduce a power envelope regulation strategy designed to constrain the variability of admittance parameters, thereby ensuring system passivity and mitigating the risk of instability. Then, sliding mode control (SMC) is employed to yield stable and robust performance. A new sliding surface is proposed based on feedback linearization, which shows improved tracking performance and stability compared to a conventional sliding surface. The effectiveness of the proposed sliding surface and associated control is theoretically validated. Notably, our modified sliding surface works universally, regardless of the order of the desired admittance equation. The trade-off between system chattering and robustness is effectively managed using a variable–boundary approach, which dynamically adjusts system constraints to optimize performance. In addition, a control algorithm combining acceleration feedback and sliding mode is proposed, showing improved robustness and tracking accuracy performance compared with applying the proposed SMC algorithm exclusively. The efficacy of these methodologies is substantiated through numerical simulations and empirical experiments.

Eye-Tracking in Physical Human-Robot Interaction: Mental Workload and Performance Prediction.

In Physical Human-Robot Interaction (pHRI), the need to learn the robot's motor-control dynamics is associated with increased cognitive load. Eye-tracking metrics can help understand the dynamics of fluctuating mental workload over the course of learning. The aim of this study was to test eye-tracking measures' sensitivity and reliability to variations in task difficulty, as well as their performance-prediction capability, in physical human-robot collaboration tasks involving an industrial robot for object comanipulation. Participants (9M, 9F) learned to coperform a virtual pick-and-place task with a bimanual robot over multiple trials. Joint stiffness of the robot was manipulated to increase motor-coordination demands. The psychometric properties of eye-tracking measures and their ability to predict performance was investigated. Stationary Gaze Entropy and pupil diameter were the most reliable and sensitive measures of workload associated with changes in task difficulty and learning. Increased task difficulty was more likely to result in a robot-monitoring strategy. Eye-tracking measures were able to predict the occurrence of success or failure in each trial with 70% sensitivity and 71% accuracy. The sensitivity and reliability of eye-tracking measures was acceptable, although values were lower than those observed in cognitive domains. Measures of gaze behaviors indicative of visual monitoring strategies were most sensitive to task difficulty manipulations, and should be explored further for the pHRI domain where motor-control and internal-model formation will likely be strong contributors to workload. Future collaborative robots can adapt to human cognitive state and skill-level measured using eye-tracking measures of workload and visual attention.

Teleoperated Surgical Robot with Adaptive Interactive Control Architecture for Tissue Identification.

The remote perception of teleoperated surgical robotics has been a critical issue for surgeons in fulfilling their remote manipulation tasks. In this article, an adaptive teleoperation control framework is proposed. It provides a physical human-robot interaction interface to enhance the ability of the operator to intuitively perceive the material properties of remote objects. The recursive least square (RLS) is adopted to estimate the required human hand stiffness that the operator can achieve to compensate for the contact force. Based on the estimated stiffness, a force feedback controller is designed to avoid the induced motion and to convey the haptic information of the slave side. The passivity of the proposed teleoperation system is ensured by the virtual energy tank. A stable contact test validated that the proposed method achieved stable contact between the slave robot and the hard environment while ensuring the transparency of the force feedback. A series of human subject experiments was conducted to empirically verify that the proposed teleoperation framework can provide a more smooth, dexterous, and intuitive user experience with a more accurate perception of the mechanical property of the interacted material on the slave side, compared to the baseline method. After the experiment, the design idea about the force feedback controller of the bilateral teleoperation is discussed.