Shared Control Framework Research Articles

Guiding real-world reinforcement learning for in-contact manipulation tasks with Shared Control Templates

The requirement for a high number of training episodes has been a major limiting factor for the application of Reinforcement Learning (RL) in robotics. Learning skills directly on real robots requires time, causes wear and tear and can lead to damage to the robot and environment due to unsafe exploratory actions. The success of learning skills in simulation and transferring them to real robots has also been limited by the gap between reality and simulation. This is particularly problematic for tasks involving contact with the environment as contact dynamics are hard to model and simulate. In this paper we propose a framework which leverages a shared control framework for modeling known constraints defined by object interactions and task geometry to reduce the state and action spaces and hence the overall dimensionality of the reinforcement learning problem. The unknown task knowledge and actions are learned by a reinforcement learning agent by conducting exploration in the constrained environment. Using a pouring task and grid-clamp placement task (similar to peg-in-hole) as use cases and a 7-DoF arm, we show that our approach can be used to learn directly on the real robot. The pouring task is learned in only 65 episodes (16 min) and the grid-clamp placement task is learned in 75 episodes (17 min) with strong safety guarantees and simple reward functions, greatly alleviating the need for simulation.

Sky-Worker: a heterogeneous dual-arm robot with dynamic authority assignment for live-line working

Purpose The current difficulties of distribution network working robots are mainly in the performance and operation mode. On the one hand, high-altitude power operation tasks require high load-carrying capacity and dexterity of the robot; on the other hand, the fully autonomous mode is uncontrollable and the teleoperation mode has a high failure rate. Therefore, this study aims to design a distribution network operation robot named Sky-Worker to solve the above two problems. Design/methodology/approach The heterogeneous arms of Sky-Worker are driven by hydraulics and electric motors to solve the contradiction between high load-carrying capacity and high flexibility. A human–robot collaborative shared control architecture is built to realize real-time human intervention during autonomous operation, and control weights are dynamically assigned based on energy optimization. Findings Simulations and tests show that Sky-Worker has good dexterity while having a high load capacity. Based on Sky-Worker, multiuser tests and practical application experiments show that the designed shared-control mode effectively improves the success rate and efficiency of operations compared with other current operation modes. Practical implications The designed heterogeneous dual-arm distribution robot aims to better serve distribution line operation tasks. Originality/value For the first time, the integration of hydraulic and motor drives into a distribution network operation robot has achieved better overall performance. A human–robot cooperative shared control framework is proposed for remote live-line working robots, which provides better operation results than other current operation modes.

A Closed-Loop Shared Control Framework for Legged Robots

Shared control, as a combination of human and robot intelligence, has been deemed as a promising direction toward complementing the perception and learning capabilities of legged robots. However, previous works on human–robot control for legged robots are often limited to simple tasks, such as controlling movement direction, posture, or single-leg motion, yet extensive training of the operator is required. To facilitate the transfer of human intelligence to legged robots in unstructured environments, this article presents a user-friendly closed-loop shared control framework. The main novelty is that the operator only needs to make decisions based on the recommendations of the autonomous algorithm, without having to worry about operations or consider contact planning issues. Specifically, a rough navigation path from the operator is smoothed and optimized to generate a path with reduced traversing cost. The traversability of the generated path is assessed using fast Monte Carlo tree search, which is subsequently fed back through an intuitive image interface and force feedback to help the operator make decisions quickly, forming a closed-loop shared control. The simulation and hardware experiments on a hexapod robot show that the proposed framework gives full play to the advantages of human–machine collaboration and improves the performance in terms of learning time from the operator, mission completion time, and success rate than comparison methods.

Practical Preassigned Fixed-Time Fuzzy Control for Teleoperation System Under Scheduled Shared-Control Framework

Practical Preassigned Fixed-Time Fuzzy Control for Teleoperation System Under Scheduled Shared-Control Framework

Shared control for human–machine co-driving vehicles based on constraint-following approach

To ensure the human–machine co-driving vehicle path-tracking accuracy, as well as to diminish the effects of vehicle underactuated characteristics and undesirable driver operation, a parallel shared control framework is proposed in this paper. By structuring a rational driving weights allocation module, the framework enables cooperative driving between the driver and automated system. First, for the front steering vehicle, a robust path-tracking controller is proposed using the constraint-following approach. The system uncertainty is decomposed into matched and non-matched uncertainties. As the non-matched portions are orthogonal to the constraint-following geometric space, they will not affect control performance. Therefore, the constraint-following control (CFC) is designed to handle the path-tracking equality constraint, initial state deviations, and matched uncertainties. Second, a fuzzy rule-based driving weights allocation module is proposed. According to the lateral deviation and the relative accuracy of the steering angle, the allocation module adjusts the driving weight of the driver and automated system to improve the vehicle condition. Finally, the simulation results show that the framework can effectively reduce the emergence caused by the driver or automated system, and significantly improve the path-tracking performance. Additionally, as the controller exhibits sufficient robustness, it can handle multiple sources of uncertainty in vehicle systems.

A Shared Control Approach Based on First-Order Dynamical Systems and Closed-Loop Variable Stiffness Control

In this paper, we present a novel learning-based shared control framework. This framework deploys first-order Dynamical Systems (DS) as motion generators providing the desired reference motion, and a Variable Stiffness Dynamical Systems (VSDS) (Chen et al. 2021) for haptic guidance. We show how to shape several features of our controller in order to achieve authority allocation, local motion refinement, in addition to the inherent ability of the controller to automatically synchronize with the human state during joint task execution. We validate our approach in a teleoperated task scenario, where we also showcase the ability of our framework to deal with situations that require updating task knowledge due to possible changes in the task scenario, or changes in the environment. Finally, we conduct a user study to compare the performance of our VSDS controller for guidance generation to two state-of-the-art controllers in a target reaching task. The result shows that our VSDS controller has the highest successful rate of task execution among all conditions. Besides, our VSDS controller helps reduce the execution time and task load significantly, and was selected as the most favorable controller by participants.

Shared autonomy and positive power control for powered exoskeletons

Powered exoskeletons have been shown to significantly reduce physical workload during occupational tasks. Due to this they have great potential impact on future labor practices. However, powered exoskeleton controllers must first be developed that are able to directly assist with task objectives and truly collaborate with users. To address this need we present a shared autonomy control framework that separates the human and exoskeleton control objectives. This allows for creating a feedback-based exoskeleton controller that will help accomplish the user’s task goals rather than just assisting in specific motions. We introduce Positive Power control for the human-based controller that is designed to allow the human to directly command work that accomplishes the desired task. While a standard robotic controller can be used for the feedback-based control, we introduce ‘acceptance’ as a measure of how well the exoskeleton’s control objective matches the human’s. Both control objectives are implemented in an optimization-based Whole-Body-Control structure. Finally, we introduce a method to update the exoskeleton’s objectives to match the humans’ such that after operating for a time the exoskeleton can learn to assist the user in accomplishing their task. This framework is implemented on a 10-DoF upper-body powered exoskeleton. The results verify that the control framework works as desired and can form a basis for developing extended methods for directly improving cooperative control for powered exoskeletons.

Cooperative Game Theory based robust shared lateral control for Driver Vehicle Interactions

Extensive manual driving is liable to attention lapses and driver exhaustion resulting in lane offset errors, driver comfort minimization and degradation in vehicle stability. On the contrary, apart from being expensive, autonomous vehicles suffer from the issue of trust, ethics and traffic mapping. In order to address the above limitations, this paper deals with the design of a Game Theoretic based Shared Control (GTSC) framework that enables an interaction between the human and the machine. The novelty lies in the generation of automatic driver automation relative weights based on the vehicle’s lateral position relative to the maximum admissible lateral offset. The interaction between the human driver and automated system based on the cooperative game theory is utilized to develop a robust shared control framework. The driver model parameters have been estimated using the experimentally obtained driver steering torque and steering wheel angle respectively through an inverse modelling procedure. The lateral dynamics of the vehicle model in the presence of crosswinds and yaw moment as disturbances are considered for the design. A higher order sliding mode observer is designed to ensure robust estimation of the unknown states. Unlike the existing studies that primarily concentrates on the robust game theoretic shared controller design, this new study broadens its focus. The study introduces an observer-based robust design for GTSC and establishes the stability of the closed-loop system. The effectiveness of this proposed framework is verified through simulations in Carsim as well as real-world testing on a Hardware-in-the-loop (HIL) platform with a human driver. The study also addresses the challenge of obstacle avoidance using the GTSC framework. Comparisons have been performed with the state-of-the-art shared control studies while considering factors like crosswinds, variation in the longitudinal velocities and alterations in the reference path to evaluate the effectiveness of the proposed shared control study. The results demonstrate the improvement in the lateral offset, driver comfort and the vehicle stability compared to the manual control of the vehicle.

Towards Human-Robot Collaborative Surgery: Trajectory and Strategy Learning in Bimanual Peg Transfer

While the traditional control of surgical robots relies on fully manual teleoperations, human-robot collaborative systems promise to address issues such as workspace constrains and laborious tasks. In particular, shared control can reduce the surgeon's workload and improve the overall surgical performance by supporting the surgeon effort during movements while keeping them in charge of complex control phases. In this letter, we propose a segmentation of the bimanual peg transfer task that alternates manual and autonomous control correspondingly. The authority allocation in this shared control framework considers both the limitation of learning-based methods and the higher dexterity of humans during physical interaction. The motion and strategies are transferred from an expert human to a da Vinci Research Kit (dVRK) [1] using an epsilon-greedy on a maximum entropy inverse reinforcement learning algorithm. The model generated enables to train an intelligent agent that can skillfully collaborate with the human operator during the surgical task. The proposed shared control framework is verified both on a virtual platform and then on a real dVRK to assess its usability and robustness. The results show that, compared to traditional teleoperation, our method can achieve faster and more consistent peg transfers. An analysis of the participants' effort also reveals a significantly lower perception of the workload.

A Human-Machine Shared Control Framework Considering Time-Varying Driver Characteristics

The uncertainties of driver's behavior seriously affect road safety and bring significant challenges to the human-machine cooperative control. This paper proposes a human-machine shared control framework considering driver's time-varying characteristics to improve the co-driving cooperation performance. Firstly, the driving intention is introduced to describe the driver's involvement level through using Gauss-Bernoulli restricted Boltzmann machine method. And the index of driving ability is proposed to evaluate driver skills based on path-tracking errors. Then, a novel human-machine authority allocation strategy is designed by combining the two driving behavior characteristics and used to construct the driver-vehicle interaction system. Subsequently, a T-S fuzzy robust state-feedback shared control system is developed considering time-varying driver behaviors and vehicle states. Finally, the proposed shared steering system is validated by the driver-in-the-loop test bench. The results show that the proposed control method can reduce human-machine conflicts and has obvious superiority in improving performance of driving comfort, path tracking, and vehicle stability for the co-driving vehicles.

Stochastic-Skill-Level-Based Shared Control for Human Training in Urban Air Mobility Scenario

This paper proposes a novel stochastic-skill-level-based shared control framework to assist human novices to emulate human experts in complex dynamic control tasks. The proposed framework aims to infer stochastic-skill-levels (SSLs) of the human novices and provide personalized assistance based on the inferred SSLs. SSL can be assessed as a stochastic variable which denotes the probability that the novice will behave similarly to experts. We propose a data-driven method which can characterize novice demonstrations as a novice model and expert demonstrations as an expert model, respectively. Then, our SSL inference approach utilizes the novice and expert models to assess the SSL of the novices in complex dynamic control tasks. The shared control scheme is designed to dynamically adjust the level of assistance based on the inferred SSL to prevent frustration or tedium during human training due to poorly imposed assistance. The proposed framework is demonstrated by a human subject experiment in a human training scenario for a remotely piloted urban air mobility (UAM) vehicle. The results show that the proposed framework can assess the SSL and tailor the assistance for an individual in real-time. The proposed framework is compared to practice-only training (no assistance) and a baseline shared control approach to test the human learning rates in the designed training scenario with human subjects. A subjective survey is also examined to monitor the user experience of the proposed framework.

A Haptic Shared Control Architecture for Tracking of a Moving Object

In this work, we propose a haptic shared control framework that integrates human and robotic control to track moving objects. Uncertainty in object detection is estimated with a Kalman filter. An obstacle avoidance algorithm is designed in conjunction with the artificial potential field method. The operator input and robot’s historical trajectory are taken into consideration and used to infer the human and robot’s control willingness according to the principle of maximum entropy. The fuzzy inference method is used to simulate human decision-making behavior and blend agents’ control. A pouring task and a docking task were designed to compare the proposed method with existing control methods. Based on the objective evaluation results, the proposed method was superior to manual control and shared control based on piecewise functions. Compared to manual control, the proposed method reduced completion time, operator input, and cognitive load by 17%, 25%, and 20% on Task1, respectively, and by 23%, 17%, and 28% on Task2, respectively. Compared to automatic control, the proposed method was more practicable although some performance indicators were less inferior. The subjective evaluation results indicated that the proposed method made it easier for agents to perceive each other’s intentions and promoted mutual trust.

A Shared Control Framework for Enhanced Grasping Performance in Teleoperation

A Shared Control Framework for Enhanced Grasping Performance in Teleoperation

Continuous shared control of a mobile robot with brain–computer interface and autonomous navigation for daily assistance

Although the electroencephalography (EEG) based brain–computer interface (BCI) has been successfully developed for rehabilitation and assistance, it is still challenging to achieve continuous control of a brain-actuated mobile robot system. In this study, we propose a continuous shared control strategy combining continuous BCI and autonomous navigation for a mobile robot system. The weight of shared control is designed to dynamically adjust the fusion of continuous BCI control and autonomous navigation. During this process, the system uses the visual-based simultaneous localization and mapping (SLAM) method to construct environmental maps. After obtaining the global optimal path, the system utilizes the brain-based shared control dynamic window approach (BSC-DWA) to evaluate safe and reachable trajectories while considering shared control velocity. Eight subjects participated in two-stage training, and six of these eight subjects participated in online shared control experiments. The training results demonstrated that naïve subjects could achieve continuous control performance with an average percent valid correct rate of approximately 97 % and an average total correct rate of over 80 %. The results of online shared control experiments showed that all of the subjects could complete navigation tasks in an unknown corridor with continuous shared control. Therefore, our experiments verified the feasibility and effectiveness of the proposed system combining continuous BCI, shared control, autonomous navigation, and visual SLAM. The proposed continuous shared control framework shows great promise in BCI-driven tasks, especially navigation tasks for brain-driven assistive mobile robots and wheelchairs in daily applications.

A Shared Autonomy Reconfigurable Control Framework for Telemanipulation of Multi-Arm Systems

Teleoperation is a widely adopted strategy to control robotic manipulators executing complex tasks that require highly dexterous movements and critical high-level intelligence. Classical teleoperation schemes are based on either joystick control, or on more intuitive interfaces which map directly the user arm motions into one robot arm's motions. These approaches have limits when the execution of a given task requires reconfigurable multiple robotic arm systems. Indeed, the simultaneous teleoperation of two or more robot arms could extend the workspace of the manipulation cell, or increase its total payload, or afford other advantages. In different phases of a reconfigurable multi-arm system, each robot could act as an independent arm, or as one of a pair of cooperating arms, or as one of the fingers of a virtual, large robot hand. This manuscript proposes a novel telemanipulation framework that enables both the individual and combined control of any number of robotic arms. Thanks to the designed control architecture, the human operator can intuitively choose the proposed control modalities and the manipulators that make the task convenient to execute through the user interface. Moreover, through the tele-impedance paradigm, the system can address complex tasks that require physical interaction by letting the robot mimic the arm impedance and position references of the human operator. The proposed framework is validated with 8 subjects controlling 4 Franka Emika Panda robots with 7-DoFs to execute a telemanipulation task. Qualitative results of the experiments show us the promising applicability of our framework.

Driver-automation shared steering control considering driver neuromuscular delay characteristics based on stackelberg game

To promote the intelligent vehicle safety and reduce the driver steering workload, stackelberg game theory is adopted to design the shared steering control strategy that takes the driver neuromuscular delay characteristics into account. First, a shared steering control framework with adjustable driving weight is proposed, and a coupling interaction model considering the driver neuromuscular delay characteristics is constructed by using the stackelberg game theory. Moreover, the driver-automation optimal control strategy is deduced theoretically when the game equilibrium is reached. Finally, simulation and virtual driving tests are carried out to verify the superiority of the proposed method. The results illustrate that the raised method can enhance the vehicle safety with low driving weight intervention, and it can achieve better auxiliary effect with less control cost. In addition, the driver-in-the-loop test results show that the proposed strategy can achieve better performance in assisting drivers with low driving skills.

Shared Control for Intelligent Vehicle Based on Handling Inverse Dynamics and Driving Intention

When we are studying the copiloting system, how to consider the driver's intention more comprehensively is a difficult issue. To this end, this paper presents a novel shared control framework based on handling inverse dynamics and driving intention for lane changing. In particular, the influence of the driver's lane-changing start point and end point is considered in the design of the shared controller. The human driver's driving intention, desired maneuver, and the start and end point are recognized. Moreover, the infinite-dimensional generalized ridge regression in reproducing kernel Hilbert spaces (RKHS) is used to deal with the lane change's start and end point prediction. Besides, a handling inverse dynamics model, including time-varying vehicle speed, is established for the automated controller, and the Multi-stage Gaussian pseudo-spectral method (GPM) is designed to solve the optimal control problem. With a shared fuzzy controller based on the state assessment, the controller generates outputs that can relieve the driver's burden. Finally, the shared control framework is validated in both overtaking and emergency obstacle avoidance scenarios. The results prove that the shared driving control framework can effectively relieve the burden on drivers and improve lane changing safety.

Convention-based Intent Integration in a Haptic Shared Control Paradigm

This paper establishes a platform to study the principles of convention formation in a haptic shared control framework wherein both humans and automation collaboratively control the steering of a semi-automated ground vehicle. We then apply these principles to determine optimal strategies for exchanging the control authority between human drivers and an automation system. In the first step, we introduce a modular structure to separate partner-specific conventions from task-dependent representations. Next, we develop a map that connects different forms of conventions to the outputs of human-automation interaction. Finally, using the convention map, we developed a reinforcement-learning-based model predictive controller to enable the automation system to learn complex policies and adapt its behavior accordingly. We applied the proposed platform to the problem of intent negotiation for resolving a conflict. Specifically, we considered a scenario where both humans and automation detect an obstacle but choose different paths to maneuver around the obstacle. The simulation results demonstrate that the convention-based handover strategies can successfully resolve a conflict and improve the performance of the human-automation teaming.

Shared control of ship autopilots and human pilots for maritime autonomous surface ship in the presence of actuator anomalies

This paper investigates the heading control problem of maritime autonomous surface ships (MASSs) in the presence of actuator anomalies. A shared control framework that includes a ship autopilot and a human pilot, is constructed to realize the accurate tracking of the time-varying command signals. Specifically, the human pilot is responsible for high-level decision making such as anomaly estimation, anomaly correction and monitoring analysis, and the ship autopilot is responsible for a low-level task of command following. With the proposed shared control framework, the ability of the ship autopilot can be significantly enhanced compared to entirely automated tracking. Through Lyapunov stability analysis, it is proven that the tracking error is ultimately bounded, while all the signals of the closed-loop system remain bounded. Finally, a simulation example is presented to prove the effectiveness of the proposed shared control architecture for MASSs under actuator anomalies.

Deep reinforcement learning for shared control of mobile robots

AbstractShared control of mobile robots integrates manual input with auxiliary autonomous controllers to improve the overall system performance. However, prior work that seeks to find the optimal shared control ratio needs an accurate human model, which is usually challenging to obtain. In this study, the authors develop an extended Twin Delayed Deep Deterministic Policy Gradient (DDPG) (TD3X)‐based shared control framework that learns to assist a human operator in teleoperating mobile robots optimally. The robot's states, shared control ratio in the previous time step, and human's control input is used as inputs to the reinforcement learning (RL) agent, which then outputs the optimal shared control ratio between human input and autonomous controllers without knowing the human model. Noisy softmax policies are developed to make the TD3X algorithm feasible under the constraint of a shared control ratio. Furthermore, to accelerate the training process and protect the robot, a navigation demonstration policy and a safety guard are developed. A neural network (NN) structure is developed to maintain the correlation of sensor readings among heterogeneous input data and improve the learning speed. In addition, an extended DAGGER (DAGGERX) human agent is developed for training the RL agent to reduce human workload. Robot simulations and experiments with humans in the loop are conducted. The results show that the DAGGERX human agent can simulate real human inputs in the worst‐case scenarios with a mean square error of 0.0039. Compared to the original TD3 agent, the TD3X‐based shared control system decreased the average collision number from 387.3 to 44.4 in a simplistic environment and 394.2 to 171.2 in a more complex environment. The maximum average return increased from 1043 to 1187 with a faster converge speed in the simplistic environment, while the performance is equally good in the complex environment because of the use of an advanced human agent. In the human subject tests, participants' average perceived workload was significantly lower in shared control than that in exclusively manual control (26.90 vs. 40.07, p = 0.013).