Safe Human-robot Interaction Research Articles

Simultaneous Motion Tracking and Joint Stiffness Control of Bidirectional Antagonistic Variable-Stiffness Actuators

Since safe human-robot interaction is naturally linked to compliance in these robots, this requirement presents a challenge for the positioning accuracy. The class of variable-stiffness robots features intrinsically soft contact behavior where the physical stiffness can even be altered during operation. Here we present a control scheme for bidirectional, antagonistic variable-stiffness actuators that achieve high-precision link-side trajectory tracking while simultaneously ensuring compliance during physical contact. Furthermore, the approach enables to regulate the pretension in the antagonism. The theoretical claims are confirmed by formal analyses of passivity during physical interaction and the proof of uniform asymptotic stability of the desired link-side trajectories. Experiments on the forearm joint of the DLR robot <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">David</i> verify the proposed approach.

Learning-Based Approach for a Soft Assistive Robotic Arm to Achieve Simultaneous Position and Force Control

Soft robotics have demonstrated great advantages in assisting elderly/disabled people during daily tasks, owing to their highly dexterous motions and safe human-robot interactions. However, simultaneously controlling the position and force of soft robots is still a challenging task due to soft actuators’ nonlinearity, system uncertainty, and high-dimensional control space. Classical control methods are usually based on first-principle/analytical models that are difficult to derive for soft robots without making significant simplifications. To overcome such control challenges, the central concept of this work is to introduce a learning-based data-driven approach. The approach employs a probabilistic model to explicitly capture system nonlinearity and uncertainty. Besides, nonparametric local learning methods are investigated to deal with redundant high-dimensional control space. The approach is applied in a soft robotic arm interacting with a manikin to simulate the bathing task. Experimental results demonstrate that the soft robotic arm could be well controlled to track the desired position and force simultaneously (maximum error of position and force is 6 mm and 0.037 N). Meanwhile, our method outperforms another typical data-driven approach (maximum error of position and force is 10 mm and 0.058 N). The results indicate that our approach is helpful for soft robots because of the physical interactions needed in assistive tasks.

Low-intensity near-infrared light-tunable gold nanorod-incorporated hydrogel actuator system for remotely controlled human–robot interface applications

Light-responsive soft actuators have recently drawn attention as the need for lightweight remotely controlled actuator systems with high environmental adaptability and safe human–robot interaction interface characteristics has increased. In this study, we focused on a near-infrared (NIR) light-tunable hydrogel soft actuator system capable of fast and precisely controllable mechanical actuation even under the irradiation of NIR light with low intensity. We designed a bilayer type hydrogel soft actuator consisting of a poly(N-isopropylacrylamide) (PNIPAAm) active layer containing gold nanorods (AuNRs) as a photothermal agent and poly(acrylamide) (PAAm) passive layer. AuNRs with an aspect ratio of 4.12 showed the largest longitudinal plasmon resonance absorption peak in the NIR 880 nm range, which was used as a photothermal agent. The AuNR-incorporated PNIPAAm/PAAm hydrogel with a bilayer structure (PNIPAAm-AuNR/PAAm) exhibited more than 70% gel fraction and temperature-sensitive swelling behaviors. In particular, rod-shaped PNIPAAm-AuNR/PAAm hydrogels showed rapid bending deformation by a photothermally induced phase transition of PNIPAAm upon 880 nm laser irradiation, but a PNIPAAm/PAAm bilayer hydrogel without AuNRs did not show a light-induced bending movement. The NIR-responsive bending actuation of the hydrogel can be precisely controlled by cross-linking density, content of photothermal agent, NIR laser intensity, and thickness ratio of the active/passive layer. It was observed that the full bending deformations of the hydrogel into a ring shape through an arc occurred within a minute under NIR irradiation with a lower intensity of less than 1.0 W/cm2. Hydrogels showed a cell viability of more than 95% in the biocompatibility test, indicating no significant cytotoxicity. Therefore, these PNIPAAm-AuNR/PAAm hydrogels are promising soft actuator materials that can be controlled by low-intensity NIR irradiation for remotely controlled human–robot interaction interface applications.

Koopman Operator-Based Knowledge-Guided Reinforcement Learning for Safe Human-Robot Interaction.

We developed a novel framework for deep reinforcement learning (DRL) algorithms in task constrained path generation problems of robotic manipulators leveraging human demonstrated trajectories. The main contribution of this article is to design a reward function that can be used with generic reinforcement learning algorithms by utilizing the Koopman operator theory to build a human intent model from the human demonstrated trajectories. In order to ensure that the developed reward function produces the correct reward, the demonstrated trajectories are further used to create a trust domain within which the Koopman operator–based human intent prediction is considered. Otherwise, the proposed algorithm asks for human feedback to receive rewards. The designed reward function is incorporated inside the deep Q-learning (DQN) framework, which results in a modified DQN algorithm. The effectiveness of the proposed learning algorithm is demonstrated using a simulated robotic arm to learn the paths for constrained end-effector motion and considering the safety of the human in the surroundings of the robot.

Co-Design of the Morphology and Actuation of Soft Robots for Locomotion

Abstract In recent years, the field of soft robotics has received considerable attention due to its potential in increasing the safety of human-robot interaction. The design of soft robots possesses great challenges. For example, the longstanding challenge of co-design morphology and actuation makes designing them by hand a trial-and-error process. Earlier work presented by the authors proposes a computational design synthesis (CDS) method for the automated design of virtual, soft locomotion robot morphologies. This work extends the CDS method for morphologies with the automated co-design of actuation. Two methods are considered. In the first method, the actuation of designs is described by parametric actuation curves (PACs) that model feedforward actuation patterns. For every morphology in the design process, a set of PACs is optimized that assumes symmetric and cyclic gaits. The second method, soft actor-critic (SAC) reinforcement learning, removes this assumption as well as models feedback control for comparison. Adding PAC optimization to the CDS method is shown to improve the performance of the resulting designs and to achieve better results within less design iterations. SAC is, however, deemed less effective, due to the need for design specific problem tuning for each new morphology. The SAC experiments also show that the best found soft robot gaits are symmetric and cyclic, although this is not a constraint in the SAC problem formulation, thus verifying the assumptions made in the PAC formulation. To validate the search space modeled in the co-design CDS method, a state-of-the-art soft robot is replicated and compared.

An Efficient Online Trajectory Generation Method Based on Kinodynamic Path Search and Trajectory Optimization for Human-Robot Interaction Safety.

With the rapid development of robot perception and planning technology, robots are gradually getting rid of fixed fences and working closely with humans in shared workspaces. The safety of human-robot coexistence has become critical. Traditional motion planning methods perform poorly in dynamic environments where obstacles motion is highly uncertain. In this paper, we propose an efficient online trajectory generation method to help manipulator autonomous planning in dynamic environments. Our approach starts with an efficient kinodynamic path search algorithm that considers the links constraints and finds a safe and feasible initial trajectory with minimal control effort and time. To increase the clearance between the trajectory and obstacles and improve the smoothness, a trajectory optimization method using the B-spline convex hull property is adopted to minimize the penalty of collision cost, smoothness, and dynamical feasibility. To avoid the collisions between the links and obstacles and the collisions of the links themselves, a constraint-relaxed links collision avoidance method is developed by solving a quadratic programming problem. Compared with the existing state-of-the-art planning method for dynamic environments and advanced trajectory optimization method, our method can generate a smoother, collision-free trajectory in less time with a higher success rate. Detailed simulation comparison experiments, as well as real-world experiments, are reported to verify the effectiveness of our method.

Expectable Motion Unit: Avoiding Hazards From Human Involuntary Motions in Human-Robot Interaction

In robotics, many control and planning schemes have been developed to ensure human physical safety in human-robot interaction. The human psychological state and the expectation towards the robot, however, are typically neglected. Even if the robot behaviour is regarded as biomechanically safe, humans may still react with a rapid involuntary motion (IM) caused by a startle or surprise. Such sudden, uncontrolled motions can jeopardize safety and should be prevented by any means. In this letter, we propose the Expectable Motion Unit (EMU), which ensures that a certain probability of IM occurrence is not exceeded in a typical HRI setting. Based on a model of IM occurrence generated through an experiment with 29 participants, we establish the mapping between robot velocity, robot-human distance, and the relative frequency of IM occurrence. This mapping is processed towards a real-time capable robot motion generator that limits the robot velocity during task execution if necessary. The EMU is combined in a holistic safety framework that integrates both the physical and psychological safety knowledge. A validation experiment showed that the EMU successfully avoids human IM in five out of six cases.

A Safe and Compliant Noncontact Interactive Approach for Wheeled Walking Aid Robot.

Aiming at promptly and accurately detecting falls and drag-to gaits induced by asynchronous human-robot movement speed during assisted walking, a noncontact interactive approach with generality, compliance and safety is proposed in this paper, and is applied to a wheeled walking aid robot. Firstly, the structure and the functions of the wheeled walking aid robot, including gait rehabilitation robot (GRR) and walking aid robot (WAR) are illustrated, and the characteristic futures of falls and the drag-to gait are shown by experiments. To obtain gait information, a multichannel proximity sensor array is developed, and a two-dimensional gait information detection system is established by combining four proximity sensors groups which are installed in the robot chassis. Additionally, a node-iterative fuzzy Petri net algorithm for abnormal gait recognition is proposed by generating the network trigger mechanism using the fuzzy membership function. It integrates the walking intention direction vector by taking gait deviation, frequency, and torso angle as input parameters of the system. Finally, to improve the compliance of the robot during human-robot interaction, a PID_SC controller is designed by integrating the gait speed compensation, which enables the WAR to track human gait closely. Abnormal gait recognition and assisted walking experiments are carried out respectively. Experimental results show that the proposed algorithm can accurately identify abnormal gaits of different groups of users with different walking habits, and the recognition rate of abnormal gait reaches 91.2%. Results also show that the developed method can guarantee safety in human robot interaction because of user gate follow-up accuracy and compliant movements. The noncontact interactive approach can be applied to robots with similar structure for usage in walking assistance and gait rehabilitation.

Implications of Spatially Constrained Bipennate Topology on Fluidic Artificial Muscle Bundle Actuation

In this paper, we investigate the design of pennate topology fluidic artificial muscle bundles under spatial constraints. Soft fluidic actuators are of great interest to roboticists and engineers, due to their potential for inherent compliance and safe human–robot interaction. McKibben fluidic artificial muscles are an especially attractive type of soft fluidic actuator, due to their high force-to-weight ratio, inherent flexibility, inexpensive construction, and muscle-like force-contraction behavior. The examination of natural muscles has shown that those with pennate fiber topology can achieve higher output force per geometric cross-sectional area. Yet, this is not universally true for fluidic artificial muscle bundles, because the contraction and rotation behavior of individual actuator units (fibers) are both key factors contributing to situations where bipennate muscle topologies are advantageous, as compared to parallel muscle topologies. This paper analytically explores the implications of pennation angle on pennate fluidic artificial muscle bundle performance with spatial bounds. A method for muscle bundle parameterization as a function of desired bundle spatial envelope dimensions has been developed. An analysis of actuation performance metrics for bipennate and parallel topologies shows that bipennate artificial muscle bundles can be designed to amplify the muscle contraction, output force, stiffness, or work output capacity, as compared to a parallel bundle with the same envelope dimensions. In addition to quantifying the performance trade space associated with different pennate topologies, analyzing bundles with different fiber boundary conditions reveals how bipennate fluidic artificial muscle bundles can be designed for extensile motion and negative stiffness behaviors. This study, therefore, enables tailoring the muscle bundle parameters for custom compliant actuation applications.

Visual servoing of continuum robots: Methods, challenges, and prospects.

Recent advancements in continuum robotics have accentuated developing efficient and stable controllers to handle shape deformation and compliance. The control of continuum robots (CRs) using physical sensors attached to the robot, particularly in confined spaces, is difficult due to their limited accuracy in three-dimensional deflections and challenging localisation. Therefore, using non-contact imaging sensors finds noticeable importance, particularly in medical scenarios. Accordingly, given the need for direct control of the robot tip and notable uncertainties in the kinematics and dynamics of CRs, many papers have focussed on the visual servoing (VS) of CRs in recent years. The significance of this research towards safe human-robot interaction has fuelled our survey on the previous methods, current challenges, and future opportunities. Beginning with actuation modalities and modelling approaches, the paper investigates VS methods in medical and non-medical scenarios. Finally, challenges and prospects of VS for CRs are discussed, followed by concluding remarks.

Serious Games Strategies With Cable-Driven Robots for Bimanual Rehabilitation: A Randomized Controlled Trial With Post-Stroke Patients.

Cable-driven robots can be an ideal fit for performing post-stroke rehabilitation due to their specific features. For example, they have small and lightweight moving parts and a relatively large workspace. They also allow safe human-robot interactions and can be easily adapted to different patients and training protocols. However, the existing cable-driven robots are mostly unilateral devices that can allow only the rehabilitation of the most affected limb. This leaves unaddressed the rehabilitation of bimanual activities, which are predominant within the common Activities of Daily Living (ADL). Serious games can be integrated with cable-driven robots to further enhance their features by providing an interactive experience and by generating a high level of engagement in patients, while they can turn monotonous and repetitive therapy exercises into entertainment tasks. Additionally, serious game interfaces can collect detailed quantitative treatment information such as exercise time, velocities, and force, which can be very useful to monitor a patient’s progress and adjust the treatment protocols. Given the above-mentioned strong advantages of both cable driven robots, bimanual rehabilitation and serious games, this paper proposes and discusses a combination of them, in particular, for performing bilateral/bimanual rehabilitation tasks. The main design characteristics are analyzed for implementing the design of both the hardware and software components. The hardware design consists of a specifically developed cable-driven robot. The software design consists of a specifically developed serious game for performing bimanual rehabilitation exercises. The developed software also includes BiEval. This specific software allows to quantitatively measure and assess the rehabilitation therapy effects. An experimental validation is reported with 15 healthy subjects and a RCT (Randomized Controlled Trial) has been performed with 10 post-stroke patients at the Physiotherapy’s Clinic of the Federal University of Uberlândia (Minas Gerais, Brazil). The RCT results demonstrate the engineering feasibility and effectiveness of the proposed cable-driven robot in combination with the proposed BiEval software as a valuable tool to augment the conventional physiotherapy protocols and for providing reliable measurements of the patient’s rehabilitation performance and progress. The clinical trial was approved by the Research Ethics Committee of the UFU (Brazil) under the CAAE N° 00914818.5.0000.5152 on plataformabrasil@saude.gov.br.

High-order control barrier functions-based impedance control of a robotic manipulator with time-varying output constraints

This paper focuses on the impedance control for robotic manipulators with time-varying output constraints. High-order control barrier functions (HoCBFs) are firstly proposed for a nonlinear system with high relative-degree time-varying constraints. Then, the HoCBFs are introduced to impedance control for robotic manipulators, where the HoCBFs are employed to avoid the violation of time-varying output constraints in Cartesian space by quadratic program (QP), and the impedance control is designed to achieve compliance for human–robot interaction (HRI). In this way, the desired trajectory within the safety-critical region can be tracked without violating the output constraints due to the controller generated from QP, and the safe HRI can also be achieved because of the usage of impedance control method. Finally, simulation tests are conducted to verify the proposed control design methods.

A Multi-Modal Sensor Array for Human-Robot Interaction and Confined Spaces Exploration Using Continuum Robots.

Safe human-robot interaction requires robots endowed with perception. This paper presents the design of a multi-modal sensory array for continuum robots, targeting operation in semi-structured confined spaces with human users. Active safety measures are enabled via sensory arrays capable of simultaneous sensing of proximity, contact, and force. Proximity sensing is achieved using time-of-flight sensors, while contact force is sensed using Hall effect sensors and embedded magnets. The paper presents the design and fabrication of these sensors, the communication protocol and multiplexing scheme used to allow an interactive rate of communication with a high-level controller, and an evaluation of these sensors for actively mapping the shape of the environment and compliance control using gestures and contact with the robot. Characterization of the proximity sensors is presented with considerations of sensitivity to lighting, color, and texture conditions. Also, characterization of the force sensing is presented. The results show that the multi-modal sensory array can enable pre and post-collision active safety measures and can also enable user interaction with the robot. We believe this new technology allows for increased safety for human-robot interaction in confined and semi-structures spaces due to its demonstrated capabilities of detecting impending collision and mapping the environment along the length of the robot. Future miniaturization of the electronics will also allow possible integration in smaller continuum and soft robots.

Minimum distance calculation using skeletal tracking for safe human-robot interaction

One of the main issues about safety in human-robot interaction is the distance calculation between robot and human. The acquisition of real-time distance information between robot and human allows the robot to modify its motion to ensure safety. This paper presents a real-time distance calculation framework that enables safe human-robot interaction. The human is tracked by an RGB-D sensor, and the joint positions of the human are acquired from the skeletal tracking algorithm. The representation of the robot and the human are formed by capsules to simplify and speed up the distance calculation. The distance calculation is implemented by using the GJK (Gilbert-Johnson-Keerthi) algorithm. The framework is validated by simulations and real experiments to show the efficiency of the framework. The results demonstrate the applicability of the framework for safe human-robot interaction applications.

Real-time motion control of robotic manipulators for safe human–robot coexistence

This paper introduces a computationally efficient control scheme for safe human–robot interaction. The method relies on the Explicit Reference Governor (ERG) formalism to enforce input and state constraints in real-time, thus ensuring that the robot can safely operate in close proximity to humans. The resulting constrained control method can steer the robot arm to the desired end-effector pose (or a steady-state admissible approximation thereof) in the presence of actuator saturation, limited joint ranges, speed limits, static obstacles, and humans. The effectiveness of the proposed solution is supported by theoretical results and numerous experimental validations on the Franka Emika Panda robotic manipulator , a commercially available collaborative 7-DOF robot arm. • Real-time predictive constrained control with Explicit Reference Governor. • Robotic arm avoids actuator saturations, limited joint and velocity constraints. • Guaranteed robot collision avoidance in static obstacle environments. • Empirical robot collision avoidance in highly dynamic environments. • Safe and performant human–robot coexistence with Franka Emika Panda.

FEM-based trajectory tracking control of a soft trunk robot

As a novel class of robots, soft robots have demonstrated many desirable mechanical properties than traditional rigid robots due to their nature of being compliant, flexible and hyper-redundant, such as great adaptability to unknown environments, safe human robot interaction (HRI), energy-saving actuation and the maneuverability to display diverse mechanical properties. However, its inherent high-DoF nature would result in some complex nonlinear behaviors, and their kinematic or dynamic models are therefore harder to deduce than the ones of conventional rigid robots. In this paper, we propose a trajectory tracking control strategy for a soft trunk robot based on Finite Element Method (FEM). We first plan a feasible trajectory for the studied robot in SOFA (a FEM-based simulator) by solving a model-prediction-control (MPC)-based optimization problem. The second step is to conduct linearization around the pre-designed trajectory, based on which an associated controller can be then developed. The detailed derivation of the mentioned work is explained accordingly. In the end, the results of experimental validation is presented to prove the feasibility of the proposed method.

State Estimation and Control with a Robust Extended Kalman Filter for a Fabric Soft Robot

Soft robots have shown great potential in safe human-robot interaction. However, their compliant nature makes posture measurement and trajectory tracking a challenging task. Accurate sensing systems such as motion capture devices aren't portable enough to use in any environment, while body-integrated soft sensors create problems like hysteresis. In this work, a robust extended Kalman filter (REKF) based sensor fusion method with an accelerometer, gyroscope, and draw wire sensor is introduced to estimate the bending angle of a fabric-based inflatable bending actuator. The REKF improves upon similar estimation techniques by ensuring a robust estimate regardless of non-linearity or uncertainty in the model. Linear Quadratic Gaussian (LQG) control is integrated with the proposed REKF to demonstrate the REKF based sensor fusion method is useful in control. The results show that the REKF based sensor fusion system presents a portable, robust, accurate estimation of the actuator's bending angle.

An Ar-Assisted Deep Reinforcement Learning-Based Approach Towards Mutual-Cognitive Safe Human-Robot Interaction

An Ar-Assisted Deep Reinforcement Learning-Based Approach Towards Mutual-Cognitive Safe Human-Robot Interaction

Joint-Space Kinematic Control of a Bionic Continuum Manipulator in Real-Time by Using Hybrid Approach

Continuum manipulators are a type of robot used for delicate applications, including safe human-robot interactions. Controlling these manipulators for an accurate trajectory, especially in the case of pneumatic actuation, is extremely challenging. Thus, this article proposes a real-time kinematic trajectory control of a pneumatically actuated multi-segment bionic continuum manipulator with a mobile base by combining a neural network and analytical model with a cascaded controller to overcome this challenge. The inverse kinematics solution of the multi-segment manipulator is developed by using a neural network and an inverse piecewise constant curvature approach. The neural network is trained by using a separate learning algorithm. Although hybrid inverse modeling gives better solutions than existing techniques, significant residual positional error of the manipulator tip remains due to inherent material hysteresis. Thus, a cascaded PI-controller is utilized to compensate for the residual positional error. The controller gains are updated in each step by predictions of the actuator length, where the Jacobian entries are computed from the neural network model. The proposed procedure is validated on Festo Didactics’ elephant trunk-like two-segment continuum manipulator, Robotino-XT. Three different cases are considered for real-time trajectory tracking, where the OptiTrack vision system is used for validation by tracking the manipulator tip pose. For the trajectory points outside the manipulator workspace, simultaneous trunk and base movements are used. In experimental validation, the proposed scheme is shown to give much reduced manipulator tip trajectory error as compared to the existing methods.

Do you feel safe with your robot? Factors influencing perceived safety in human-robot interaction based on subjective and objective measures

Safety in human-robot interaction can be divided into physical safety and perceived safety, where the latter is still under-addressed in the literature. Investigating perceived safety in human-robot interaction requires a multidisciplinary perspective. Indeed, perceived safety is often considered as being associated with several common factors studied in other disciplines, i.e., comfort, predictability, sense of control, and trust. In this paper, we investigated the relationship between these factors and perceived safety in human-robot interaction using subjective and objective measures. We conducted a two-by-five mixed-subjects design experiment. There were two between-subjects conditions: the faulty robot was experienced at the beginning or the end of the interaction. The five within-subjects conditions correspond to (1) baseline, and the manipulations of robot behaviors to stimulate: (2) discomfort, (3) decreased perceived safety, (4) decreased sense of control and (5) distrust. The idea of triggering a deprivation of these factors was motivated by the definition of safety in the literature where safety is often defined by the absence of it. Twenty-seven young adult participants took part in the experiments. Participants were asked to answer questionnaires that measure the manipulated factors after within-subjects conditions. Besides questionnaire data, we collected objective measures such as videos and physiological data. The questionnaire results show a correlation between comfort, sense of control, trust, and perceived safety. Since these factors are the main factors that influence perceived safety, they should be considered in human-robot interaction design decisions. We also discuss the effect of individual human characteristics (such as personality and gender) that they could be predictors of perceived safety. We used the physiological signal data and facial affect from videos for estimating perceived safety where participants’ subjective ratings were utilized as labels. The data from objective measures revealed that the prediction rate was higher from physiological signal data. This paper can play an important role in the goal of better understanding perceived safety in human-robot interaction.