Shared Control Architecture Research Articles

Adaptive active inceptor design under shared control architecture for nonlinear pilot-induced oscillations

This paper presents a Shared Control Architecture (SCA) between a human pilot and a smart inceptor for nonlinear Pilot Induced Oscillations (PIOs), e.g., category II or III PIOs. One innovation of this paper is that an intelligent shared control architecture is developed based on the intelligent active inceptor technique, i.e., Smart Adaptive Flight Effective Cue (SAFE-Cue). A deep reinforcement learning approach namely Deep Deterministic Policy Gradient (DDPG) method is chosen to design a gain adaptation mechanism for the SAFE-Cue module. By doing this, the gains of the SAFE-Cue will be intelligently tuned once nonlinear PIOs triggered; meanwhile, the human pilot will receive a force cue from the SAFE-Cue, and will consequently adapting his/her control policy. The second innovation of this paper is that the reward function of the DDPG based gain adaptation approach is constructed according to flying qualities. Under the premise of considering failure situation, task completion qualities and pilot workload are also taken into account. Finally, the proposed approach is validated using numerical simulation experiments with two types of scenarios: lower actuator rate limits and airframe damages. The Inceptor Peak Power-Phase (IPPP) metric is adopted to analyze the human-vehicle system simulation results. Results and analysis show that the DDPG based sharing control approach can well address nonlinear PIO problems consisting of Categories II and III PIO events.

Sky-Worker: a heterogeneous dual-arm robot with dynamic authority assignment for live-line working

Purpose The current difficulties of distribution network working robots are mainly in the performance and operation mode. On the one hand, high-altitude power operation tasks require high load-carrying capacity and dexterity of the robot; on the other hand, the fully autonomous mode is uncontrollable and the teleoperation mode has a high failure rate. Therefore, this study aims to design a distribution network operation robot named Sky-Worker to solve the above two problems. Design/methodology/approach The heterogeneous arms of Sky-Worker are driven by hydraulics and electric motors to solve the contradiction between high load-carrying capacity and high flexibility. A human–robot collaborative shared control architecture is built to realize real-time human intervention during autonomous operation, and control weights are dynamically assigned based on energy optimization. Findings Simulations and tests show that Sky-Worker has good dexterity while having a high load capacity. Based on Sky-Worker, multiuser tests and practical application experiments show that the designed shared-control mode effectively improves the success rate and efficiency of operations compared with other current operation modes. Practical implications The designed heterogeneous dual-arm distribution robot aims to better serve distribution line operation tasks. Originality/value For the first time, the integration of hydraulic and motor drives into a distribution network operation robot has achieved better overall performance. A human–robot cooperative shared control framework is proposed for remote live-line working robots, which provides better operation results than other current operation modes.

An efficient grasping shared control architecture for unpredictable and unspecified tasks.

Robot control in complex and unpredictable scenarios presents challenges such as adaptability, robustness, and human-robot interaction. These scenarios often require robots to perform tasks that involve unknown objects in unstructured environments with high levels of uncertainty. Traditional control methods, such as automatic control, may not be suitable due to their limited adaptability and reliance on prior knowledge. Human-in-the-loop method faces issues such as insufficient feedback, increased failure rates due to noise and delays, and lack of operator immersion, preventing the achievement of human-level performance. This study proposed a shared control framework to achieve a trade-off between efficiency and adaptability by combing the advantages of both teleoperation and automatic control method. The proposed approach combines the advantages of both human and automatic control methods to achieve a balance between performance and adaptability. We developed a linear model to compare three control methods and analyzed the impact of position noise and communication delays on performance. The real-world implementation of the shared control system demonstrates its effectiveness in object grasping and manipulation tasks. The results suggest that shared control can significantly improve grasping efficiency while maintaining adaptability in task execution for practical robotics applications.

A Haptic Shared Control Architecture for Tracking of a Moving Object

In this work, we propose a haptic shared control framework that integrates human and robotic control to track moving objects. Uncertainty in object detection is estimated with a Kalman filter. An obstacle avoidance algorithm is designed in conjunction with the artificial potential field method. The operator input and robot’s historical trajectory are taken into consideration and used to infer the human and robot’s control willingness according to the principle of maximum entropy. The fuzzy inference method is used to simulate human decision-making behavior and blend agents’ control. A pouring task and a docking task were designed to compare the proposed method with existing control methods. Based on the objective evaluation results, the proposed method was superior to manual control and shared control based on piecewise functions. Compared to manual control, the proposed method reduced completion time, operator input, and cognitive load by 17%, 25%, and 20% on Task1, respectively, and by 23%, 17%, and 28% on Task2, respectively. Compared to automatic control, the proposed method was more practicable although some performance indicators were less inferior. The subjective evaluation results indicated that the proposed method made it easier for agents to perceive each other’s intentions and promoted mutual trust.

The Classification and New Trends of Shared Control Strategies in Telerobotic Systems: A Survey.

Shared control, which permits a human operator and an autonomous controller to share the control of a telerobotic system, can reduce the operator's workload and/or improve performances during the execution of tasks. Due to the great benefits of combining the human intelligence with the higher power/precision abilities of robots, the shared control architecture occupies a wide spectrum among telerobotic systems. Although various shared control strategies have been proposed, a systematic overview to tease out the relation among different strategies is still absent. This survey, therefore, aims to provide a big picture for existing shared control strategies. To achieve this, we propose a categorization method and classify the shared control strategies into 3 categories: Semi-Autonomous control (SAC), State-Guidance Shared Control (SGSC), and State-Fusion Shared Control (SFSC), according to the different sharing ways between human operators and autonomous controllers. The typical scenarios in using each category are listed and the advantages/disadvantages and open issues of each category are discussed. Then, based on the overview of the existing strategies, new trends in shared control strategies, including the "autonomy from learning" and the "autonomy-levels adaptation," are summarized and discussed.

Shared control of ship autopilots and human pilots for maritime autonomous surface ship in the presence of actuator anomalies

This paper investigates the heading control problem of maritime autonomous surface ships (MASSs) in the presence of actuator anomalies. A shared control framework that includes a ship autopilot and a human pilot, is constructed to realize the accurate tracking of the time-varying command signals. Specifically, the human pilot is responsible for high-level decision making such as anomaly estimation, anomaly correction and monitoring analysis, and the ship autopilot is responsible for a low-level task of command following. With the proposed shared control framework, the ability of the ship autopilot can be significantly enhanced compared to entirely automated tracking. Through Lyapunov stability analysis, it is proven that the tracking error is ultimately bounded, while all the signals of the closed-loop system remain bounded. Finally, a simulation example is presented to prove the effectiveness of the proposed shared control architecture for MASSs under actuator anomalies.

An Energy-Based Approach for n-d.o.f. Passive Dual-User Haptic Training Systems

SUMMARYThis paper introduces a dual-user training system whose design is based on an energetic approach. This kind of system is useful for supervised hands-on training where a trainer interacts with a trainee through two haptic devices, in order to practice on a manual task performed on a virtual or teleoperated robot (e.g., for an Minimally Invasive Surgery (MIS) task in a surgical context). This paper details the proof of stability of an Energy Shared Control (ESC) architecture we previously introduced for one degree of freedom (d.o.f.) devices. An extension to multiple degrees of freedom is proposed, along with an enhanced version of the Adaptive Authority Adjustment function. Experiments are carried out with 3 d.o.f. haptic devices in free motion as well as in contact contexts in order to show the relevance of this architecture.

A multilateral impedance-controlled system for haptics-enabled surgical training and cooperation in beating-heart surgery

In this paper, an impedance-controlled multi-master/single-slave telerobotic system is developed for haptics-enabled surgical training and cooperation in beating-heart surgery. This system not only can enable automatically motion compensation for the beating heart’s motion as well as non-oscillatory force feedback to the human operators but can also enable training and cooperation for multiple users. A multi-user shared control architecture is developed, and a multilateral impedance-controlled strategy is employed for this architecture. The desired objectives of the proposed system are (a) providing position guidance to the trainees during training procedure, (b) providing force feedback to all human operators (trainer and trainees) regardless of their levels of authority over the slave robot, (c) motion compensation for the heart’s motion, and (d) reflecting only the non-oscillatory force portion of the slave-heart tissue interaction force to all human operators. To this end, virtual fixtures and a dominance factor are introduced, and a reference impedance model with adjusted parameters is designed for each master or slave robot. The proposed impedance-based control methodology is evaluated experimentally. The experimental results demonstrated that the proposed method could be used for surgical training and cooperation in beating-heart surgery by providing appropriate position guidance and environmental force feedback to the human operators.

A Haptic Shared-Control Architecture for Guided Multi-Target Robotic Grasping.

Although robotic telemanipulation has always been a key technology for the nuclear industry, little advancement has been seen over the last decades. Despite complex remote handling requirements, simple mechanically linked master-slave manipulators still dominate the field. Nonetheless, there is a pressing need for more effective robotic solutions able to significantly speed up the decommissioning of legacy radioactive waste. This paper describes a novel haptic shared-control approach for assisting a human operator in the sort and segregation of different objects in a cluttered and unknown environment. A three-dimensional scan of the scene is used to generate a set of potential grasp candidates on the objects at hand. These grasp candidates are then used to generate guiding haptic cues, which assist the operator in approaching and grasping the objects. The haptic feedback is designed to be smooth and continuous as the user switches from a grasp candidate to the next one, or from one object to another one, avoiding any discontinuity or abrupt changes. To validate our approach, we carried out two human-subject studies, enrolling 15 participants. We registered an average improvement of 20.8%, 20.1%, and 32.5% in terms of completion time, linear trajectory, and perceived effectiveness, respectively, between the proposed approach and standard teleoperation.

Shared control architectures for vehicle steering

Various schemes for sharing control between a human driver and automation system have been proposed, each with the aim of freeing attention while supporting smooth transitions of authority when the control challenge exceeds driver or automation capabilities. When sharing control of a vehicle, it can be expected that the driver develops internal models of the automation and its capabilities and assumes or assigns intent to its actions. In this paper, we develop system models of input mixing and haptic shared control to describe the communication channels open to the driver for monitoring automation behavior and determining automation intent. We pay particular attention to haptic (torque) feedback in the axis of steering that functions both to carry information and to couple the dynamics of the driver’s backdrivable arms and hands with the dynamics of the automation system and vehicle. We assess the various feedback loops present for their promise to reduce cognitive load while maintaining situation awareness. An interpretation of the backdrivable biomechanics and automation impedance in terms of potential wells produces insight into the structured information available through haptic feedback and the internal models that the driver uses to predict automation behavior. The constructed models make explicit the additional communication channels open between human and automation in haptic shared control relative to input mixing.

Tele-Operated MRI-Guided Needle Insertion for Prostate Interventions

Prostate cancer is one of the leading causes of death in men. Prostate interventions using magnetic resonance imaging (MRI) benefits from high tissue contrast if compared to other imaging modalities. The Minimally Invasive Robotics In An MRI environment (MIRIAM) robot is an MRI-compatible system able to steer different types of needles towards a point of interest using MRI guidance. However, clinicians can be reluctant to give the robot total control of the intervention. This work integrates a haptic device in the MIRIAM system to allow input from the clinician during the insertion. A shared control architecture is achieved by letting the clinician control the insertion depth via the haptic device, while the robotic system controls the needle orientation. The clinician receives haptic feedback based on the insertion depth and tissue characteristics. Four control laws relating the motion of the master robot (haptic device) to the motion of the slave robot (MIRIAM robot) are presented and evaluated. Quantitative and qualitative results from 20 human subjects demonstrate that the squared-velocity control law is the most suitable option for our application. Additionally, a pre-operative target localization algorithm is presented in order to provide the robot with the target location. The target localization and reconstruction algorithm are validated in phantom and patient images with an average dice similarity coefficient (DSC) of 0.78. The complete system is validated through experiments by inserting a needle towards a target within the MRI scanner. Four human subjects perform the experiment achieving an average targeting error of 3.4[Formula: see text]mm.

Shared Control Between Adaptive Autopilots and Human Operators for Anomaly Mitigation

Abstract As aerial vehicles become more autonomous, and guidance and navigation systems become increasingly network-centric, there is a need to consider a swift response to the growing forms of anomalies that may occur in these systems. We introduce a shared decision-making and control framework between a human operator and an adaptive autopilot, where the human operator plays a supervisory role and the adaptive autopilot retains the responsibility for low-level regulation and command tracking tasks. The human operator provides key inputs based on a higher-level perception of the anomaly, such as an increased lag in response to command inputs, which are then used by the adaptive autopilot in a suitable manner. The resulting shared control architecture is demonstrated on an unmanned aerial vehicle, whose actuators abruptly change from first-order to second-order due to an anomaly.

Discussion of “A Review of Intent Detection, Arbitration, and Communication Aspects of Shared Control for Physical Human–Robot Interaction” (Losey, D. P., McDonald, C. G., Battaglia, E., and O'Malley, M. K., 2018, ASME Appl. Mech. Rev., 70(1), p. 010804)

A unifying description of the shared control architecture within the field of physical human–robot interaction (pHRI) facilitates the education of those being introduced to the field and the framing of new contributions to it. The authors' review of shared control within pHRI proposes such a unifying framework composed of three pillars. First, intent detection addresses the robot's interpretation of human goals, representing one-way communication. Second, arbitration manages the respective roles of the human and robot in the shared control. Third, feedback is the mechanism by which the robot returns information to the human, representing one-way communication in the opposite direction. Interpreting existing contributions through the lens of this framework brings out the importance of mechanical design, modeling, and state-based control.

Obstacle avoidance control of a human-in-the-loop mobile robot system using harmonic potential fields

SUMMARYThis paper considers applications where a human agent is navigating a semi-autonomous mobile robot in an environment with obstacles. The human input to the robot can be based on a desired navigation objective, which may not be known to the robot. Additionally, the semi-autonomous robot can be programmed to ensure obstacle avoidance as it navigates the environment. A shared control architecture can be used to appropriately fuse the human and the autonomy inputs to obtain a net control input that drives the robot. In this paper, an adaptive, near-continuous control allocation function is included in the shared controller, which continuously varies the control effort exerted by the human and the autonomy based on the position of the robot relative to obstacles. The developed control allocation function facilitates the human to freely navigate the robot when away from obstacles, and it causes the autonomy control input to progressively dominate as the robot approaches obstacles. A harmonic potential field-based non-linear sliding mode controller is developed to obtain the autonomy control input for obstacle avoidance. In addition, a robust feed-forward term is included in the autonomy control input to maintain stability in the presence of adverse human inputs, which can be critical in applications such as to prevent collision or roll-over of smart wheelchairs due to erroneous human inputs. Lyapunov-based stability analysis is presented to guarantee finite-time stability of the developed shared controller, i.e., the autonomy guarantees obstacle avoidance as the human navigates the robot. Experimental results are provided to validate the performance of the developed shared controller.

Enhanced transparency dual-user shared control teleoperation architecture with multiple adaptive dominance factors

In traditional dual-user shared control teleoperation, the users’ operational transparency is influenced by the dominance factorα. Especially when α = 1 and α = 0 the trainer or trainee cannot receive any information form the slave side. For enlarging the control flexibility and enhancing the system transparency, a novel method with multiple adaptive dominance factors is proposed in this paper. An ideal application of this method is on-line operation and supervision. The supervisor and operator can adjust the factors and switch the operation roles and states freely. Once the slave robot handled by the operator diverges from the planned path, the dominance factors will change adaptive to limit the operator’s movement. The adaptive principles are concluded from the dynamic performance measured by the measuring functions. The conclusions suggest that the varying range of the system dynamic performance is wider than the traditional method. In addition, considering the time delays between the slave and master sides, we proved the system stability conditions covering all the range of dominance factors. Finally, we make a discussion of the applying area of the novel shared control architecture.

Bumpless Reengagement Using Shared Control between Human Pilot and Adaptive Autopilot

In this paper, we consider a flight control problem subjected to actuator anomalies. We propose a shared control architecture that includes actions of both a human pilot and an adaptive autopilot that achieves bumpless performance while retaining a Capacity for Maneuver (CfM) following the anomaly. The shared controller distributes the critical tasks of anomaly perception and adaptation between a human pilot and an adaptive autopilot to best utilize their individual strengths. The human pilot’s expertise in detection and perception of anomaly is utilized to determine an initial estimate of the plant parameters. An adaptive autopilot then carries out further tuning of the controller to minimize the tracking error while retaining a certain CfM. The shared controller is evaluated in simulation study of an F-8 aircraft under actuator anomalies and shown to be advantageous compared to a fixed-gain fault tolerant controller.

Design and evaluation of a trilateral shared-control architecture for teleoperated training robots.

Multilateral teleoperated robots can be used to train humans to perform complex tasks that require collaborative interaction and expert supervision, such as laparoscopic surgical procedures. In this paper, we explain the design and performance evaluation of a shared-control architecture that can be used in trilateral teleoperated training robots. The architecture includes dominance and observation factors inspired by the determinants of motor learning in humans, including observational practice, focus of attention, feedback and augmented feedback, and self-controlled practice. Toward the validation of such an architecture, we (1) verify the stability of a trilateral system by applying Llewellyn's criterion on a two-port equivalent architecture, and (2) demonstrate that system transparency remains generally invariant across relevant observation factors and movement frequencies. In a preliminary experimental study, a dyad of two human users (one novice, one expert) collaborated on the control of a robot to follow a trajectory. The experiment showed that the framework can be used to modulate the efforts of the users and adjust the source and level of haptic feedback to the novice user.

Analysis of Coupled Stability in Multilateral Dual-User Teleoperation Systems

In this paper, we set out a framework for the analysis of coupled stability in dual-user linear teleoperation systems. An extension of the Zeheb–Walach (ZW) criteria for absolute stability of an $n$ -port network will be stated and proven. While the original theorem states conditions for asymptotic stability of a network terminated by passive impedances, the extended version allows for poles on the imaginary axis, which makes it applicable to a larger class of systems, such as robotic applications with position feedback. The extended theorem includes conditions on the Laurent expansion of the elements and the principal minors of the network immittance matrix. A novel dual-user shared control paradigm, realizing a three-way gearbox mechanism, is presented. A numerical analysis of absolute stability of the three-port network, representing the shared control architecture, demonstrates the effectiveness of the extended Zeheb–Walach method.

A Framework for Unconditional Stability Analysis of Multimaster/Multislave Teleoperation Systems

A novel robust stability analysis framework is presented for unconditional stability analysis of multimaster/multislave teleoperation systems. Unlike the unconditional stability criterion for single-user systems, the newly proposed criteria for unconditional stability of multimaster/multislave teleoperation systems depend on the multiport network parameters and the port terminations. In addition to the analytical solution, the graphical demonstration of the unconditional stability region facilitates the analysis of coupled stability against variations in the dynamics of the environments and operators, even when they behave actively. The proposed robust stability analysis framework is examined on two multilateral shared control architectures that were previously developed for dual-user teleoperation systems.

Shared control architecture based on RFID to control a robot arm using a spontaneous brain–machine interface

This paper describes a shared control architecture combining a Brain–Machine Interface (BMI) with Radio-frequency Identification (RFID) technology to control a robot arm in pick and place operations. A non-invasive spontaneous BMI capable of distinguishing between three different mental tasks has been designed. Using the BMI, the user can control the robot in order to perform complex actions (e.g. pick and place operations). RFID tags have been placed in the experimental setup to give information about the position of the objects in the scene. With this information, the user is able to pick and place the objects with a robot arm by performing simple commands: move left, move right, pick or place, with the only help of the BMI. Four volunteers have successfully controlled the robot arm, and time and accuracy have been measured.