Human-robot Interaction Control Research Articles

Asymmetric Integral Barrier Lyapunov Function-Based Human–Robot Interaction Control for Human-Compliant Space-Constrained Muscle Strength Training

Asymmetric Integral Barrier Lyapunov Function-Based Human–Robot Interaction Control for Human-Compliant Space-Constrained Muscle Strength Training

Physical Human–Robot Interaction Control of an Upper Limb Exoskeleton With a Decentralized Neuroadaptive Control Scheme

Physical Human–Robot Interaction Control of an Upper Limb Exoskeleton With a Decentralized Neuroadaptive Control Scheme

Passive Model-Predictive Impedance Control for Safe Physical Human–Robot Interaction

Various cognitive systems have been designed to model the position and stiffness profiles of human behavior and then to drive robots by mimicking the human’s behavior to accomplish physical human-robot interaction tasks through a properly designed impedance controller. However, some studies have shown that variable stiffness parameters of the impedance controller can cause the violation of the passivity constraint of the robot states, and make the robot’s stored energy exceed the external energy injected from the human user, thus leading to the unsafe human-robot interaction. To solve this problem, this paper proposes a novel passive model predictive impedance control method including two control loops. In the bottom-loop of the proposed controller, the robot is driven by a variable impedance controller to achieve the desired compliant interaction behavior. In the top-loop of the proposed controller, the model predictive control (MPC) is used to ensure that the robot states satisfy the passivity constraint by calculating a complementary torque to limit the stored energy of the robot. The passivity of the closed-loop robot system and the feasibility of MPC are guaranteed by theoretical analysis, ensuring the safety of the robotic movement in the human-robot interaction. The effectiveness of the proposed method is demonstrated by the simulation and experiment on the Franka Emika Panda robot.

Research on mode adjustment control strategy of upper limb rehabilitation robot based on fuzzy recognition of interaction force

In the process of robot-assisted training for upper limb rehabilitation, a passive training strategy is usually used for stroke patients with flaccid paralysis. In order to stimulate the patient's active rehabilitation willingness, the rehabilitation therapist will use the robot-assisted training strategy for patients who gradually have the ability to generate active force. This study proposed a motor function assessment technology for human upper-limb based on fuzzy recognition on interaction force and human-robot interaction control strategy based on assistance-as-needed. A passive training mode based on the calculated torque controller and an assisted training mode combined with the potential energy field were designed, and then the interactive force information collected by the three-dimensional force sensor during the training process was imported into the fuzzy inference system, the degree of active participation σ was proposed, and the corresponding assisted strategy algorithms were designed to realize the adaptive adjustment of the two modes. The significant correlation between the degree of active participation σ and the surface electromyography signals (sEMG) was found through the experiments, and the method had a shorter response time compared to a control strategy that only adjusted the mode through the magnitude of interaction force, making the robot safer during the training process.

Design of Human-Machine Integration System to Meet Diverse Interactive Tasks

A flexible production model in which humans and robots collaborate has become an urgent need for industrial production. Therefore, research on human-machine integration systems has received more and more attention in recent years. Human-computer interaction is widely used in industrial production and entertainment life, such as industrial control, VR games, etc. In this article, we designed and implemented a human-machine fusion system that meets different interactive tasks. Human movement estimation and robot interactive control are the key technologies. First, we propose a human posture solution method based on information fusion of multiple cameras and inertial measurement units. This method constructs an optimization problem, fuses the 2D joint detection information under multiple cameras and the measurement information of the worn inertial measurement unit, and optimizes the estimation of human kinematic posture, which improves the problems of incomplete posture information and sensitivity to noise under a single sensor. Therefore, this method improves the accuracy of pose estimation. Secondly, combining the kinematic characteristics of the robot and the characteristics of human-computer interaction, we designed a robot control strategy based on target point tracking and model predictive control, so that the robot can adapt to dynamic environments and different interaction needs by adjusting control parameters. At the same time, our design ensures the safety of robots and operators. Finally, we conducted human-computer interaction experiments such as action following, item delivery, and active obstacle avoidance. The experimental results show the effectiveness and reliability of the designed robot interaction system in a human-machine fusion environment.

Multimodal fusion and human-robot interaction control of an intelligent robot.

Introduction: Small-scaled robotic walkers play an increasingly important role in Activity of Daily Living (ADL) assistance in the face of ever-increasing rehab requirements and existing equipment drawbacks. This paper proposes a Rehabilitation Robotic Walker (RRW) for walking assistance and body weight support (BWS) during gait rehabilitation. Methods: The walker provides the patients with weight offloading and guiding force to mimic a series of the physiotherapist's (PT's) movements, and creates a natural, comfortable, and safe environment. This system consists of an omnidirectional mobile platform, a BWS mechanism, and a pelvic brace to smooth the motions of the pelvis. To recognize the human intentions, four force sensors, two joysticks, and one depth-sensing camera were used to monitor the human-machine information, and a multimodal fusion algorithm for intention recognition was proposed to improve the accuracy. Then the system obtained the heading angle E, the pelvic pose F, and the motion vector H via the camera, the force sensors, and the joysticks respectively, classified the intentions with feature extraction and information fusion, and finally outputted the motor speed control through the robot's kinematics. Results: To validate the validity of the algorithm above, a preliminary test with three volunteers was conducted to study the motion control. The results showed that the average error of the integral square error (ISE) was 2.90 and the minimum error was 1.96. Discussion: The results demonstrated the efficiency of the proposed method, and that the system is capable of providing walking assistance.

Repetitive Impedance Learning-Based Physically Human-Robot Interactive Control.

Model-based impedance learning control can provide variable impedance regulation for robots through online impedance learning without interaction force sensing. However, the existing related results only guarantee the closed-loop control systems to be uniformly ultimately bounded (UUB) and require the human impedance profiles being periodic, iteration-dependent, or slowly varying. In this article, a repetitive impedance learning control approach is proposed for physical human-robot interaction (PHRI) in repetitive tasks. The proposed control is composed of a proportional-differential (PD) control term, an adaptive control term, and a repetitive impedance learning term. Differential adaptation with projection modification is designed for estimating robotic parameters uncertainties in the time domain, while fully saturated repetitive learning is proposed for estimating time-varying human impedance uncertainties in the iterative domain. Uniform convergence of tracking errors is guaranteed by the PD control and the use of projection and full saturation in the uncertainties estimation and is theoretically proved based on a Lyapunov-like analysis. In impedance profiles, the stiffness and damping are composed of an iteration-independent term and an iteration-dependent disturbance, which are estimated by repetitive learning and compressed by the PD control, respectively. Therefore, the developed approach can be applied to the PHRI where iteration-dependent disturbances exist in the stiffness and damping. The control effectiveness and advantages are validated by simulations on a parallel robot in a repetitive following task.

Machine Learning in Robot-Assisted Upper Limb Rehabilitation: A Focused Review

Robot-assisted rehabilitation, which can provide repetitive, intensive and high-precision physics training, has a positive influence on motor function recovery of stroke patients. Current robots need to be more intelligent and more reliable in clinical practice. Machine learning algorithms (MLAs) are able to learn from data and predict future unknown conditions, which is of benefit to improve the effectiveness of robot-assisted rehabilitation. In this paper, we conduct a focused review on machine learning-based methods for robot-assisted upper limb rehabilitation. Firstly, the current status of upper rehabilitation robots is presented. Then, we outline and analyze the designs and applications of MLAs for upper limb movement intention recognition, human-robot interaction control and quantitative assessment of motor function. Meanwhile, we discuss the future directions of MLAs-based robotic rehabilitation. This review article provides a summary of MLAs for robotic upper limb rehabilitation and contributes to the design and development of future advanced intelligent medical devices.

Performance-Based Iterative Learning Control for Task-Oriented Rehabilitation: A Pilot Study in Robot-Assisted Bilateral Training

Active participation from human subjects can enhance the effectiveness of robot-assisted rehabilitation. Developing interactive control strategies for customized assistance is therefore essential for encouraging human-robot engagement. However, existing human-robot interactive control strategies lack precise evaluation indicators with effective convergence method to steadily and rapidly customize appropriate assistance during task-oriented training. This study proposes a performance-based iterative learning control algorithm for robot-assisted training, which aims at providing subject-specific robotic assistance to encourage active participation. Three performance indicators based on a Fugl-Meyer Assessment (FMA) regression model are introduced to associate clinical scales with robot-based measures, and a fuzzy logic is employed for comprehensive performance evaluation. To increase efficient training time, a piecewise learning rate based iterative law is applied to quickly converge to a subject-specific control parameter session by session. The proposed strategy is preliminarily estimated for a case of bilateral upper limb training with an end-effector based robotic system. Experimental results with human subjects indicate that the proposed strategy can obtain appropriate parameters after only several iterations and adapt to random perturbations (like muscle fatigue).

Physical Human–Robot Interaction Control of Variable Stiffness Exoskeleton With sEMG-Based Torque Estimation

Robotic exoskeleton assistance is an effective method for the treatment of patients with movement disorders. Firstly, this paper presents a variable stiffness knee exoskeleton robot (VS-KExo), which can independently control stiffness and position and has a large stiffness range under low preload. Then, combined with the designed variable stiffness exoskeleton, a physical human-robot interaction (pHRI) control scheme based on joint torque estimation is proposed. Different from previous studies, this method uses the mechanical properties of the exoskeleton to achieve pHRI without complex control algorithms and force/torque sensors. Furthermore, the joint torque is estimated based on the surface electromyography signal (sEMG) and the Hill-based muscle model, and the exoskeleton can realize assist-as-needed (AAN) function: 1) when the subject trains with less effort, the exoskeleton maintains a high output physical impedance to provide high tracking accuracy; and 2) when the subject trains with greater effort, the exoskeleton maintains a low output impedance to provide high physical compliance. Finally, we conducted experiments on three healthy subjects and two subjects with lower limb motor dysfunction to verify the effectiveness of the torque estimation method and the pHRI control scheme.

Neuromuscular State Estimation Via Space-by-Time Neural Signal Decomposition

Abstract Robotic exoskeletons for the hand are being explored to improve health, safety, and physical performance. However, much research effort is needed to establish reliable models of human behavior for effective human–robot interaction control. In this work, surface electromyography is used to measure and model muscle activity of healthy participants performing quasi-isometric and dynamic hand exercises. Non-negative matrix tri-factorization is used to extract hidden neuromuscular parameters encoded in spatial and temporal muscle synergies, which are used to estimate probabilistic linear models of intent, effort, and fatigue. This paper thereby presents steps toward reliable modeling of nonlinear time-varying hand neuromuscular dynamics for intuitive and robust human–robot interaction.

Task-oriented human-robot interaction control of a robotic glove utilizing forearm electromyography

Task-oriented myoelectric assistive control of a robotic glove based on forearm electromyography for practical grasping tasks is investigated in this work. Three grasping functions are considered: human intent detection of grasping tasks, grasping mode recognition, and grasping force control. Different combinations of forearm electromyographic signals are adopted for the three functions. Firstly, the overall electromyographic signal is used to trigger the whole grasping task. Secondly, a novel Long Short-Term Memory network is utilized to classify various grasping modes, including pinch grasping and palmar grasping, by analyzing eight-channel electromyographic signals. Thirdly, two-channel proportional control of grasping force using electromyographic flexor/extensor signals is adopted for the robotic glove such that the human hand can relax during the grasping task, while the robotic glove maintains the grasping force. To this end, finite state machine based hierarchical control architecture is proposed for the whole grasping task. Experiments are conducted to validate the proposed task-oriented assistive control method, and the results clearly demonstrates the potential of the proposed method in rehabilitation therapy.

Analysis of Motor Control and Learning in Human-Robot Interaction during Game Guided Movements.

Robotic devices can be used in upper limb rehabilitation in order to help the total or partial functional recovery. Robots can perform repetitive activities for a long period of time, which may be beneficial for rehabilitation processes. In this context, this study uses a bi-manual robotic device to investigate motor learning and control for the upper limbs among different game guided tasks, and inspect the user's grip force exerted in response to perturbations. The robotic device resembles a bicycle handlebar, instrumented with load cells to measure torques and grip forces. It is equipped with a DC motor to apply external torques to the guiding system. A game was developed containing in-game and physical perturbations to the natural movement of the handlebar. Tests were carried out with 16 healthy subjects that were instructed to move the handlebar guiding a character displayed on the screen with the objective of collecting tokens to get the higher score in the game. During the trials, corresponding data from the game and the load cells were collected and used to infer the learning process, the mean error in the trajectory and the variations in the force applied to the handles of the handlebar. Analyses showed that there was learning in the first repetitions, and the learning was retained further. The higher values of the grip force occurred when there was a physical perturbation to the handlebar's natural movement. The larger errors in the trajectories occurred immediately after the perturbations ended. In conclusion, there was a performance improvement, probably related to learning. The increase of the mean error at the transitions of the perturbations indicates the need for adaptation to the new conditions of the task.

RESEARCH ON HUMAN–ROBOT PHYSICAL INTERACTION CONTROL BASED ON ADAPTIVE IMPEDANCE CONTROL

An adaptive impedance control method based on dexterity for compliant interaction is proposed for the problem of compliance and motion performance of the human’s healthy side interacting with the manipulator in bilateral mirror rehabilitation motion-assisted training, and constructs a local virtual force field to hinder the movement of the manipulator to the low motion performance region, thus improving the motion performance of the manipulator. Firstly, the dynamic model of the manipulator and the human–robot interaction model based on impedance control are established. Then, a dexterity index based on the condition number is established, and an adaptive impedance control method based on the dexterity is proposed to construct a local virtual force field to hinder the movement of the manipulator to the low motion performance region. Finally, the effectiveness of the proposed method is demonstrated by experiments. The results have shown that the adaptive impedance control method based on dexterity can construct a local virtual force field, which can constrain the motion of the manipulator and keep the robot with good motion performance. It also laid the foundation for the training strategy of bilateral mirror rehabilitation.

An artificial systems, computational experiments and parallel execution‐based surface electromyogram‐driven anti‐disturbance zeroing neurodynamic strategy for upper limb human‐robot interaction control

AbstractIn recent years, intelligent robots are extensively applied in the field of the industry and intelligent rehabilitation, wherein the human‐robot interaction (HRI) control strategy is a momentous part that needs to be ameliorated. Specially, the efficacy and robustness of the HRI control algorithm in the presence of unknown external disturbances deserve to be addressed. To deal with these urgent issues, in this study, artificial systems, computational experiments and a parallel execution intelligent control framework are constructed for the HRI control. The upper limb‐robotic exoskeleton system is re‐modelled as an artificial system. Depending on surface electromyogram‐based subject's active motion intention in the practical system, a non‐convex function activated anti‐disturbance zeroing neurodynamic (NC‐ADZND) controller is devised in the artificial system for parallel interaction and HRI control with the practical system. Furthermore, the linear activation function‐based zeroing neurodynamic (LAF‐ZND) controller and proportional‐derivative (posterior deltoid (PD)) controller are presented and compared. Theoretical results substantiate the global convergence and robustness of the proposed controller in the presence of different external disturbances. In addition, the simulation results verify that the NC‐ADZND controller is better than the LAF‐ZND and the PD controllers in respect of convergence order and anti‐disturbance characteristics.

An Artificial Intelligence‐Motivated Skin‐Like Optical Fiber Tactile Sensor

Soft and stretchable tactile sensors have received extensive attention for their potential applications in wearables, human–robot interaction, and intelligent robots. Herein, inspired by the functions of skin somatosensory signal generation and processing, an artificial intelligence‐motivated skin‐like optical fiber tactile (SOFT) sensor is proposed. It features multifunctional touch interaction capabilities including tactile amplitude and position and tensile strain. Four fiber Bragg gratings (FBGs) are embedded in a skin‐like three‐layer laminate structure of the SOFT sensor, forming a flexible tactile sensing array with a stretchability larger than 20%. Fusing the two‐level cascaded neural network, the position and magnitude of the contact force can be distinguished simultaneously. The recognition accuracy for contact position is up to 92.41% and the error is less than 4.2% within the force range of 0–3.5 N. Several SOFT sensor‐based interactive applications including pressure password interface and music playback are achieved by combining the artificial intelligence spatiotemporal dynamic logic analysis. Furthermore, the sensor is also capable of complex scenes involving tension and tactile sensing, such as dexterous hand perception and human–robot interaction control. This work provides novel insights into artificial intelligence‐based integrated skin that shows broad promise in intelligent prosthetics and bionic robotic.

Hand Gesture Recognition and Control for Human-Robot Interaction Using Deep Learning

This paper introduces a real-time system for recognizing hand gestures using Python and OpenCV, centred on a Convolutional Neural Network (CNN) model. The primary objective of this study is to address the challenge of recognizing hand gestures in varied and complex environments. The proposed approach employs several image and video processing techniques, including data augmentation and feature extraction, to segment the hand region and extract relevant features. The system’s performance is significantly improved by adding to the original training dataset, resulting in 5,000 images with 500 images per gesture, as shown by the evaluation metrics indicating a substantial increase in accuracy from 96.9% to 99.2%. This paper aims to provide feasible and economical solutions for utilizing robots in industrial settings, while also proposing future research possibilities for enhancing human-robot interaction through methods such as incorporating hand gesture recognition.

Sensorless Variable Admittance Control for Human–Robot Interaction of a Dual-Arm Social Robot

Sensorless Variable Admittance Control for Human–Robot Interaction of a Dual-Arm Social Robot

Interaction Task Motion Learning for Human–Robot Interaction Control

Traditional applications of robot manipulators are mainly limited to tracking control tasks, in which the desired objectives are specified as desired positions or trajectories. In such applications, a convenient and easy way of programming robots is the traditional teach-and-playback method. However, advances in sensing and robotic technologies have led to the requirements of more demanding tasks, in which robots may need to interact with a human or follow the human’s instructions in performing a sequence of more complex tasks. In such applications, it is not sufficient to just learn the positions or motion and play it back using a robot controller. In this article, a task learning approach is proposed for human–robot interaction systems, where a set of interaction behaviors is formulated and solved by specifying the task requirements in terms of potential energy. The motion behaviors demonstrated by humans can thus be acquired by the robot by seeking the appropriate task parameters of the dynamic potential energy function. To play back and combine the tasks in a sequential way by using a single controller, a new robot controller is also proposed. Lyapunov-like analysis is adopted to guarantee the stability of the control system, and experimental results are given to validate the performance of the proposed learning and control strategies.

Noise-tolerant zeroing neurodynamic algorithm for upper limb motion intention-based human–robot interaction control in non-ideal conditions

Human–robot interaction control, a crucial technology in industrial manufacturing and rehabilitation robot, has been widely explored in recent years. However, there are still some problems in the existing strategy need to be solved for the rehabilitation robot: First, the human–robot confrontation (HRC) in motion control affects the information interaction. Second, in non-ideal conditions, the safety and accuracy of the control system will be disturbed by external unknown factors. Third, the existing continuous-time controllers are hard to be applied to hardware platforms. In this paper, an upper limb-exoskeleton coupling dynamics model based on human active motion intention is established to overcome HRC, and promote active and natural human–robot interaction control. Specifically, the Elman neural network (ENN) model is devised and utilized to estimate human active motion intention using the surface electromyography (sEMG) signals. On the basis of human active motion intention, the discrete-time controller predicated on noise-tolerant zeroing neurodynamic model (termed as NTZND controller) is proposed for constructing the discrete-time human–robot interaction control system (DTHRICS) under non-ideal conditions. In addition, the zeroing neurodynamic (ZND) controller, the gradient neural network (GNN) and proportional–derivative (PD) controllers are designed for comparisons. Theoretical analyses demonstrate that the 0-stability, consistency and convergence of the designed DTHRICS. The performance indicators root mean square errors (RMSEs) and coefficient of determination (R2) of the DTHRICS with different controllers are analyzed and compared under ideal and non-ideal conditions. Ultimately, numerical results illustrate that the NTZND-based DTHRICS (with the lowest performance RMSEs →0 and the highest performance R2→1, p<0.05) is significantly superior to the DTHRICS with the ZND, the GNN and the PD controllers in terms of control accuracy, noise-tolerant property and real time performance, while utilized for upper limb multi-joint motion control under non-ideal conditions.