Human-robot Cooperative Control Research Articles

Evaluation of a machine-learning-driven active-passive upper-limb exoskeleton robot: Experimental human-in-the-loop study.

Evaluating exoskeleton actuation methods and designing an effective controller for these exoskeletons are both challenging and time-consuming tasks. This is largely due to the complicated human-robot interactions, the selection of sensors and actuators, electrical/command connection issues, and communication delays. In this research, a test framework for evaluating a new active-passive shoulder exoskeleton was developed, and a surface electromyography (sEMG)-based human-robot cooperative control method was created to execute the wearer's movement intentions. The hierarchical control used sEMG-based intention estimation, mid-level strength regulation, and low-level actuator control. It was then applied to shoulder joint elevation experiments to verify the exoskeleton controller's effectiveness. The active-passive assistance was compared with fully passive and fully active exoskeleton control using the following criteria: (1) post-test survey, (2) load tolerance duration, and (3) computed human torque, power, and metabolic energy expenditure using sEMG signals and inverse dynamic simulation. The experimental outcomes showed that active-passive exoskeletons required less muscular activation torque (50%) from the user and reduced fatigue duration indicators by a factor of 3, compared to fully passive ones.

A framework and method for Human-Robot cooperative safe control based on digital twin

Human-robot collaboration (HRC) combines the robot’s mechanical properties and predictability with human experience, logical thinking, and strain capabilities to alleviate production efficiency. However, ensuring the safety of the HRC process in-real time has become an urgent issue. Digital twin extends functions of virtual models in the design phase of the physical counterpart in the production phase through virtual-real interactive feedback, data fusion analysis, advanced computational features, etc. This paper proposes an HRC safety control framework and corresponding method based on the digital twin. In the design phase, virtual simulation and virtual reality technology are integrated to construct virtual twins of various HRC scenarios for testing and analyzing potential safety hazards. In the production phase, the safety distance between humans and robots of the HRC scene is monitored and calculated by an iterative algorithm according to machine vision and a convolutional neural network. Finally, the virtual twin is driven based on real-scene data, real-time online visual monitoring, and optimization of the HRC’s overall process. A case study using ABB-IRB1600 is presented to verify the feasibility of the proposed approach.

Gait Rehabilitation and Locomotion Support System Using a Distributed Controlled Robot System

In this paper, we proposed a distributed controlled gait rehabilitation and locomotion support system through human-robot cooperative control, and combination of two cane-type walking support robots (left and right) and one wheelchair robot to support the walking and locomotion of a person in need of walking assistance. The proposed system can realize five types of motion support from gait training to daily motion support, using three types of motion support modes with a distributed robot control system comprising up to three robots to support the user’s independence for walking and moving. The cane-type walking support robot moved in response to the manipulation force applied to the robot by the user, and can realize walking/movement support in all directions through the omnidirectional traveling part. In addition, the height of the robot can be adjusted according to the user’s physique, and the motion characteristics can be set according to the user’s walking ability. The wheelchair robot has a seat that can be raised, lowered, and tilted to provide standing assistance for the user and mobility support as an electric wheelchair. In this study, we developed a prototype of the proposed system and demonstrated its feasibility for five types of assistive actions in experiments with healthy subjects.

Human-robot cooperative control system based on serial elastic actuator bowden cable drive in ExoArm 7-DOF upper extremity exoskeleton

The exoskeleton of the upper limb is an external parallel kinematic chain to the human arm. The device is designed to apply a specific torque of interaction to the human body resulting from bilateral teleoperation or rehabilitation. Its task is to work comfortably with the human arm. This paper presents the structure of the developed device and the control system of its joints. The construction of the joints’ drive system was performed based on the Bowden cable transmission. Based on the Bowden cable flexibility, it is possible to control the generated drive torque following the serial elastic actuator concept. The article presents joint control methods minimizing the influence of friction in the closed-loop Bowden cable conduit system. We also present the methods of estimating the torque of interaction with the operator based on the ExoArm 7-DOF exoskeleton dynamics model.

Human-robot cooperative control based on sEMG for the upper limb exoskeleton robot

An exoskeleton robot is a mechanical structure that integrates with the exterior of the human body to improve the wearer’s muscular power. The key to ensure performances and comfort of the system is human–robot cooperation. This paper proposes a human–robot cooperative control method based on sEMG (surface Electromyography) signals to drive a pneumatic upper limb exoskeleton to act in accordance with the wearer’s motion intentions. The intended movement information of the human is estimated by combining the regression method with the classification method. Based on the joint torque estimation model which is originated from the Hill-type musculoskeletal model, the regression method is used to estimate the joint’s desired torque by merging the sEMG signal with the joint angle. To avoid shaking and keep the robot’s limbs in the static condition, a classification method with the support vector machine is developed to find out the joint state that the human intends to keep. It was then applied to the exoskeleton’s elbow joint flexion and extension movement experiments to verify the controller’s effectiveness. The experimental results demonstrate that the controller can estimate human’s motion intention accurately and is appropriate for the human–robot collaboration.

Knee exoskeleton enhanced with artificial intelligence to provide assistance-as-needed.

Robotic therapy is a useful method applied during rehabilitation of stroke patients (to regain motor functions). To ensure active participation of the patient, assistance-as-needed is provided during robotic training. However, most existing studies are based on a predetermined desired trajectory, which significantly limits the use of this method for more complex scenarios. In this paper, artificial intelligence (AI) agents are introduced to enhance the robot so that a knee exoskeleton can be autonomously controlled. A new assist-as-needed (AAN) method is proposed, where the subjects and agents cooperatively control movements. An electromyographic (EMG)-controlled knee exoskeleton with an interesting screen game is developed. Two different AI agents, modular pipeline and deep Q-network, are introduced; both can control the exoskeleton to play the screen game independently. The human-robot cooperative control is studied with two different assistant strategies, i.e., fixed assistant ratio and AAN. Eight healthy subjects participated in the initial experiment, and four assistant modes were studied. The game scores obtained by the two agents were significantly higher than those obtained by healthy subjects (EMG control), indicating that using the agents to assist stroke rehabilitation is possible. The AAN method demonstrated a better performance than the fixed assistant ratio method, indicated by the higher integral muscle activation level and participant score. Compared to a fully active control (EMG control) and fully fixed guidance (AI control), human-robot cooperative control had significantly higher integral muscle activation levels, i.e., the subjects were more involved and motivated during training. Using AI agents to power rehabilitation robots is a promising way to realize AAN rehabilitation.

Human–Robot Cooperative Control Based on Virtual Fixture in Robot-Assisted Endoscopic Sinus Surgery

In endoscopic sinus surgery, the robot assists the surgeon in holding the endoscope and acts as the surgeon’s third hand, which helps to reduce the surgeon’s operating burden and improve the quality of the operation. This paper proposes a human–robot cooperative control method based on virtual fixture to realize accurate and safe human–robot interaction in endoscopic sinus surgery. Firstly, through endoscopic trajectory analysis, the endoscopic motion constraint requirements of different surgical stages are obtained, and three typical virtual fixtures suitable for endoscopic sinus surgery are designed and implemented. Based on the typical virtual fixtures, a composite virtual fixture is constructed, and then the overall robot motion constraint model is obtained. Secondly, based on the obtained robot motion constraint model, a human–robot cooperative control method based on virtual fixture is proposed. The method adopts admittance control to realize efficient human–robot interaction between the surgeon and robot during the surgery; the virtual fixture is used to restrain and guide the motion of the robot, thereby ensuring motion safety of the robot. Finally, the proposed method is evaluated through a robot-assisted nasal endoscopy experiment, and the result shows that the proposed method can improve the accuracy and safety of operation during endoscopic sinus surgery.

Intelligent Control for Human-Robot Cooperation in Orthopedics Surgery.

Cooperation between surgeon and robot is one of the key technologies that limit the robot to be widely used in orthopedic clinics. In this study, the evolution of human-robot cooperation methods and the control strategies for typical human-robot cooperation in robot-assisted orthopedics surgery were reviewed at first. Then an intelligent admittance control method, which combines the fuzzy model reference learning control with the virtual constraint control, is proposed to solve the requirements of intuitive human-robot interaction during orthopedics surgery. That is, a variable damping parameter model of the admittance control based on fuzzy model learning control algorithm is introduced to make the robot move freely by using the reference model of surgeon's motion equation with the minimum jerk trajectory. And the virtual constraint control method based on the principle of virtual fixture is adopted to make the robot move within the pre-defined area so as to perform more safe surgery. The basic principle and its realization of this intelligent control method are described in details. At last, a test platform is built based on our designed 6 DOF articulated robot. Experiments of safety and precision on acrylic model with this method show that the robot has the ability of better intuitive interaction and the high precision. And the pilot experiment of bone tumor resection on sawbone model shows the effectiveness of this method.

Experiments on Human-in-the-Loop Coordination for Multirobot System With Task Abstraction

This paper presents the experimental validation of a multiple mobile robot system with human-in-the-loop coordination. It has been proposed that the control framework for networked autonomous robots can be augmented by artificial functions for task abstraction, so that a human operator is able to operate a group of robots remotely. By utilizing the redundancy of a multirobot system with task-space control, the group of mobile robots, in addition to achieving the missions tele-controlled by the human, can perform added tasks simultaneously. In this paper, the human-robot cooperative control system was validated experimentally through a robotic manipulator and a group of mobile robots by taking into account communication delays. Multirobot coverage control, formation control, and cooperative transportation were demonstrated, to validate the performance and efficiency of the robotic control system with human-in-the-loop coordination.

Kinematic modeling and control for human-robot cooperation considering different interaction roles

SUMMARYThis paper presents a novel approach for the description of physical human-robot interaction (pHRI) tasks that involve two-arm coordination, and where tasks are described by the relative pose between the human hand and the robot hand. We develop a unified kinematic model that takes into account the human-robot system from a holistic point of view, and we also propose a kinematic control strategy for pHRI that comprises different levels of shared autonomy. Since the kinematic model takes into account the complete human-robot interaction system and the kinematic control law is closed loop at the interaction level, the kinematic constraints of the task are enforced during its execution. Experiments are performed in order to validate the proposed approach, including a particular case where the robot controls the human arm by means of functional electrical stimulation (FES), which may potentially provide useful solutions for the interaction between assistant robots and impaired individuals (e.g., quadriplegics and hemiplegics).

Human-robot cooperative control based on pHRI (Physical Human-Robot Interaction) of exoskeleton robot for a human upper extremity

This paper proposes a human-robot cooperative control of exoskeleton robot assisting muscle strength of a human upper extremity when lifting or transporting heavy objects. When a human wears a robot, the motions of the human and robot generate interaction, which is called HRI (Human-Robot Interaction). To generate reference motion from the interaction force, a pHRI model was developed using virtual mechanical impedance, and an experimental method to determine the impedance parameters of the pHRI model was proposed. The controller was developed in such a way that the desired motion will be controlled using dynamic model-based compensation. To verify the proposed control method, it was applied to an exoskeleton robot with 6-DOF for both arms. Motion-following-performance experiment and muscle-strength-assisting-effect experiment were conducted using this robot. Experimental results, the wearer of the exoskeleton robot can handle a small force was the heavy object.

A Comparative Evaluation of Control Interfaces for a Robotic-Aided Endoscopic Capsule Platform

Wireless capsule endoscopy offers significant advantages compared with traditional endoscopic procedures, since it limits the invasiveness of gastrointestinal tract screening and diagnosis. Moreover, active locomotion devices would allow endoscopy to be performed in a totally controlled manner, avoiding failures in the correct visualization of pathologies. Previous works demonstrated that magnetic locomotion through a robotic-aided platform would allow us to reach this goal reliably. In this paper, the authors present a comparative evaluation of control methodologies and user interfaces for a robotic-aided magnetic platform for capsule endoscopy, controlled through human-robot cooperative and teleoperated control algorithms. A detailed statistical analysis of significant control parameters was performed: teleoperated control is the more reliable control approach, and a serial kinematic haptic device results as the most suitable control interface to perform effective robotic-aided endoscopic procedures.

Using adjustable autonomy and human–machine cooperation to make a human–machine system resilient – Application to a ground robotic system

This study concerns autonomous ground vehicles performing missions of observation or surveillance. These missions are accomplished under the supervision of human operators, who can also remotely control the unmanned vehicle. This kind of human–machine system is likely to face perturbations in a dynamic natural environment. However, human operators are not able to manage perturbations due to overload. The objective of this study is to provide such systems with ways to anticipate, react and recover from perturbations. In other words, these works aim at improving system resilience so that it can better manage perturbations. This paper presents a model of human–robot cooperative control that helps to improve the resilience of the human–machine system by making the level of autonomy adjustable. A formalism of agent autonomy is proposed according to the semantic aspects of autonomy and the agent’s activity levels. This formalism is then used to describe the activity levels of the global human–machine system. Hierarchical decision-making methods and planning algorithms are also proposed to implement these levels of activity. Finally, an experimental illustration on a micro-world is presented in order to evaluate the feasibility and application of the proposed model.

Reaction Force/Torque Sensing in a Master-Slave Robot System without Mechanical Sensors

In human-robot cooperative control systems, force feedback is often necessary in order to achieve high precision and high stability. Usually, traditional robot assistant systems implement force feedback using force/torque sensors. However, it is difficult to directly mount a mechanical force sensor on some working terminals, such as in applications of minimally invasive robotic surgery, micromanipulation, or in working environments exposed to radiation or high temperature. We propose a novel force sensing mechanism for implementing force feedback in a master-slave robot system with no mechanical sensors. The system consists of two identical electro-motors with the master motor powering the slave motor to interact with the environment. A bimanual coordinated training platform using the new force sensing mechanism was developed and the system was verified in experiments. Results confirm that the proposed mechanism is capable of achieving bilateral force sensing and mirror-image movements of two terminals in two reverse control directions.