Shared Control Method Research Articles

Shared Control of an Electric Wheelchair Considering Physical Functions and Driving Motivation.

Individuals with severe physical impairments have difficulties operating electric wheelchairs (EWs), especially in situations where fine steering abilities are required. Automatic driving partly solves the problem, although excessive reliance on automatic driving is not conducive to maintaining their residual physical functions and may cause more serious diseases in the future. The objective of this study was to develop a shared control system that can be adapted to different environments by completely utilizing the operating ability of the user while maintaining the motivation of the user to drive. The operating characteristics of individuals with severe physical impairments were first analyzed to understand their difficulties when operating EWs. Subsequently, a novel reinforcement learning-based shared control method was proposed to adjust the control weight between the user and the machine to meet the requirements of fully exploiting the operating abilities of the users while assisting them when necessary. Experimental results showed that the proposed shared control system gradually adjusted the control weights between the user and the machine, providing safe operation of the EW while ensuring full use of the control signals from the user. It was also found that the shared control results were deeply affected by the types of users.

A Teleoperation Framework for Mobile Robots Based on Shared Control

Mobile robots can complete a task in cooperation with a human partner. In this letter, a hybrid shared control method for a mobile robot with omnidirectional wheels is proposed. A human partner utilizes a six degrees of freedom haptic device and electromyography (EMG) signals sensor to control the mobile robot. A hybrid shared control approach based on EMG and artificial potential field is exploited to avoid obstacles according to the repulsive force and attractive force and to enhance the human perception of the remote environment based on force feedback of the mobile platform. This shared control method enables the human partner to tele-control the mobile robot's motion and achieve obstacles avoidance synchronously. Compared with conventional shared control methods, this proposed one provides a force feedback based on muscle activation and drives the human partners to update their control intention with predictability. Experimental results demonstrate the enhanced performance of the mobile robots in comparison with the methods in the literature.

A Muscle Teleoperation System of a Robotic Rollator Based on Bilateral Shared Control

The approach that achieves the teleoperation between human muscle signals and the mobile robot is increasingly applied to transfer human muscle stiffness to enhance robotic performance. In this paper, we develop a mobile rollator control system applying a muscle teleoperation interface and a shared control method to enhance the obstacle avoidance in an effective way. In order to control intuitively, haptic feedback is utilized in the teleoperation interface and is integrated with EMG stiffness to provide a large composition force. Then the composition force is implemented with an artificial potential field method to keep the robotic rollator away from the obstacle in advance. This algorithm is superior to the traditional APF algorithm regardless of the required time and trajectory length. The experimental results demonstrate the effectiveness of the proposed muscle teleoperation system.

Driver-Automation Cooperation Oriented Approach for Shared Control of Lane Keeping Assist Systems

This paper presents a novel shared control concept for lane keeping assist (LKA) systems of intelligent vehicles. The core idea is to combine system perception with robust control so that the proposed strategy can successfully share the control authority between human drivers and the LKA system. This shared control strategy is composed of two parts, namely an operational part and a tactical part. Two local optimal-based controllers with two predefined objectives (i.e., lane keeping and conflict management) are designed in the operational part. The control supervisor in the tactical part aims to provide a decision-making signal which allows for a smooth transition between two local controllers. The control design is based on a human-in-the-loop vehicle system to improve the mutual driver-automation understanding, thus reducing or avoiding the conflict. The closed-loop stability of the whole driver-vehicle system can be rigorously guaranteed using the Lyapunov stability argument. In particular, the control design is formulated as an LMI optimization which can be easily solved with numerical solvers. The effectiveness of the proposed shared control method is clearly demonstrated through various hardware experiments with human drivers.

A robotic shared control teleoperation method based on learning from demonstrations

In teleoperation, the operator is often required to command the motion of the remote robot and monitor its behavior. However, such an interaction demands a heavy workload from a human operator when facing with complex tasks and dynamic environments. In this article, we propose a shared control method to assist the operator in the manipulation tasks to reduce the workload and improve the efficiency. We adopt a task-parameterized hidden semi-Markov model to learn a manipulation skill from several human demonstrations. We utilize the learned model to predict the manipulation target given the current observed robotic motion trajectory and subsequently estimate the desired robotic motion given the current input of the operator. The estimated robotic motion is then utilized to correct the input of the operator to provide manipulation assistance. In addition, a set of virtual reality devices are used to capture the operator’s motion and display the vision feedback from the remote site. We evaluate our approach through two manipulation tasks with a dual-arm robot. The experimental results show the effectiveness of the proposed method.

Fused Fuzzy Petri Nets: A Shared Control Method for Brain–Computer Interface Systems

It is hard to grasp objects based on brain–computer interface (BCI) by brain-actuated robot arm and hand due to its high degree of freedom. Shared control strategy and hybrid BCI are research trends to solve this control problem of brain-actuated discrete event system. We propose a new shared control method based on fused fuzzy Petri nets (PNs) for combining the robot automatic control (AC) and the brain-actuated control. This method takes the advantages of both fuzzy control and PNs such as easy modeling, robustness, and effectiveness. Both MATLAB simulation test and Barrett robot hand practical experiments show that the proposed method performs much better than AC or BCI control independently. In the online BCI practical experiment, the user successfully control the Barrett robot hand to grasp object avoiding obstacle in whole ten random scenes by our shared control method and hybrid BCI. Compared with BCI control, the user needs not to synchronously work according to the specific paradigm in the whole process. Meanwhile, our method improves safety and robustness by comparing with AC.

Asynchronous brain-computer interface shared control of robotic grasping

The control of a high Degree of Freedom (DoF) robot to grasp a target in three-dimensional space using Brain-Computer Interface (BCI) remains a very difficult problem to solve. Design of synchronous BCI requires the user perform the brain activity task all the time according to the predefined paradigm; such a process is boring and fatiguing. Furthermore, the strategy of switching between robotic auto-control and BCI control is not very reliable because the accuracy of Motor Imagery (MI) pattern recognition rarely reaches 100 % . In this paper, an asynchronous BCI shared control method is proposed for the high DoF robotic grasping task. The proposed method combines BCI control and automatic robotic control to simultaneously consider the robotic vision feedback and revise the unreasonable control commands. The user can easily mentally control the system and is only required to intervene and send brain commands to the automatic control system at the appropriate time according to the experience of the user. Two experiments are designed to validate our method: one aims to illustrate the accuracy of MI pattern recognition of our asynchronous BCI system; the other is the online practical experiment that controls the robot to grasp a target while avoiding an obstacle using the asynchronous BCI shared control method that can improve the safety and robustness of our system.

Shared control-based bimanual robot manipulation.

Human-centered environments provide affordances for and require the use of two-handed, or bimanual, manipulations. Robots designed to function in, and physically interact with, these environments have not been able to meet these requirements because standard bimanual control approaches have not accommodated the diverse, dynamic, and intricate coordinations between two arms to complete bimanual tasks. In this work, we enabled robots to more effectively perform bimanual tasks by introducing a bimanual shared-control method. The control method moves the robot's arms to mimic the operator's arm movements but provides on-the-fly assistance to help the user complete tasks more easily. Our method used a bimanual action vocabulary, constructed by analyzing how people perform two-hand manipulations, as the core abstraction level for reasoning about how to assist in bimanual shared autonomy. The method inferred which individual action from the bimanual action vocabulary was occurring using a sequence-to-sequence recurrent neural network architecture and turned on a corresponding assistance mode, signals introduced into the shared-control loop designed to make the performance of a particular bimanual action easier or more efficient. We demonstrate the effectiveness of our method through two user studies that show that novice users could control a robot to complete a range of complex manipulation tasks more successfully using our method compared to alternative approaches. We discuss the implications of our findings for real-world robot control scenarios.

Model Predictive-Based Shared Control for Brain-Controlled Driving

Using brain signals rather than limbs to drive a vehicle can help persons with disabilities to extend their movement range and, thus, to improve their self-independence. However, the driving performance of brain-controlled vehicles (BCVs) is poor. In this paper, to improve the performance of BCVs, we propose a new shared control method based on the model predictive control (MPC) strategy. Particularly, to maintain the maximum control authority of brain-control drivers while ensuring the safety of BCVs, the MPC controller is designed by introducing a penalty on the deviation from drivers output in the cost function and setting safety constraints. Driver-and-hardware-in-the-loop experiments are conducted under two road-keeping scenarios and one obstacle-avoidance scenario with different subjects to validate the proposed method. The results demonstrate the effectiveness of the proposed method in avoiding roadway departures and obstacles while maintaining the control authority of users.

Human–machine shared control for lane departure assistance based on hybrid system theory

To improve the human–machine cooperation of lane departure assistance system (LDAS), a human–machine shared control strategy based on hybrid system theory was proposed. By considering vehicle’s discrete and continuous states and time-varying longitudinal speed, the hybrid system was formalized as hybrid automaton, and the shared control strategy was built to govern the human–machine interaction. Robust gain-scheduling energy-to-peak method was adopted to design the assistance system controller. The D-stability of the system was also studied and guaranteed by solving the linear matrix inequality (LMI). The proposed human–machine shared control method was evaluated via the co-simulation of CarSim/Simulink and the hardware-in-loop (HIL) experiment. The results showed that the proposed approach can effectively keep the vehicle in lane and a good human–machine coordination was demonstrated.

Combining controls can improve power in two-stage association studies

BackgroundHigh dimensional case control studies are ubiquitous in the biological sciences, particularly genomics. To maximise power while constraining cost and to minimise type-1 error rates, researchers typically seek to replicate findings in a second experiment on independent cohorts before proceeding with further analyses. This can be an expensive procedure, particularly when control samples are difficult to recruit or ascertain; for example in inter-disease comparisons, or studies on degenerative diseases.ResultsThis paper presents a method in which control (or case) samples from the discovery cohort are re-used in a replication study. The theoretical implications of this method are discussed and simulated genome-wide association study (GWAS) tests are used to compare performance against the standard approach in a range of circumstances.Using similar methods, a procedure is proposed for ‘partial replication’ using a new independent cohort consisting of only controls. This methods can be used to provide some validation of findings when a full replication procedure is not possible.The new method has differing sensitivity to confounding in study cohorts compared to the standard procedure, which must be considered in its application. Type-1 error rates in these scenarios are analytically and empirically derived, and an online tool for comparing power and error rates is provided.ConclusionsIn several common study designs, a shared-control method allows a substantial improvement in power while retaining type-1 error rate control. Although careful consideration must be made of all necessary assumptions, this method can enable more efficient use of data in GWAS and other applications.

Driver observation and shared vehicle control: supporting the driver on the way back into the control loop

Abstract In the near future, drivers of automated cars will still have to take over from time to time at short notice. Current control systems implement a hard switch, disabling the automation all at once. However, studies show that the driver’s ability to take over depends on his last activity. We therefore propose a system that uses camera based observation of the driver to assess the situation and to predict transition times. We combine this with a control system that uses a cooperative shared control method to support the driver in takeover situations and allows him to adjust safely to the current situation. We present our first steps towards this goal and show both how the behavior of the driver in the interior can be assessed and how a cooperative control transfer can be implemented. We further point out the necessary steps to implement the proposed system and give a first impression of the performance via simulation.

Sensor Reduction for Driver-Automation Shared Steering Control via an Adaptive Authority Allocation Strategy

This paper presents a new shared control method for lane keeping assist (LKA) systems of intelligent vehicles. The proposed method allows the LKA system to effectively share the control authority with a human driver by avoiding or minimizing the conflict situations between these two driving actors. To realize the shared control scheme, the unpredictable driver-automation interaction is explicitly taken into account in the control design via a fictive driver activity variable. This latter is judiciously introduced into the driver–road–vehicle system to represent the driver's need for assistance in accordance with his/her real-time driving activity. Using Lyapunov stability arguments, Takagi–Sugeno fuzzy model-based design conditions are derived to handle not only the time-varying driver activity variable, but also a large variation range of vehicle speed. Both simulation and hardware experiments are presented to demonstrate that the proposed control strategy together with a linear matrix inequality design formulation provide an effective tool to deal with the challenging shared steering control issue. In particular, a fuzzy output feedback control scheme is exploited to achieve the shared control goal without at least two important vehicle sensors. These physical sensors are widely employed in previous works to measure the lateral speed and the steering rate for the control design and real-time implementation. The successful results of this idea of sensor-reduction control has an obvious interest from practical viewpoint.

Prediction of user preference over shared-control paradigms for a robotic wheelchair.

The design of intelligent powered wheelchairs has traditionally focused heavily on providing effective and efficient navigation assistance. Significantly less attention has been given to the end-user's preference between different assistance paradigms. It is possible to include these subjective evaluations in the design process, for example by soliciting feedback in post-experiment questionnaires. However, constantly querying the user for feedback during real-world operation is not practical. In this paper, we present a model that correlates objective performance metrics and subjective evaluations of autonomous wheelchair control paradigms. Using off-the-shelf machine learning techniques, we show that it is possible to build a model that can predict the most preferred shared-control method from task execution metrics such as effort, safety, performance and utilization. We further characterize the relative contributions of each of these metrics to the individual choice of most preferred assistance paradigm. Our evaluation includes Spinal Cord Injured (SCI) and uninjured subject groups. The results show that our proposed correlation model enables the continuous tracking of user preference and offers the possibility of autonomy that is customized to each user.

Shared control on lunar spacecraft teleoperation rendezvous operations with large time delay

Teleoperation could be used in space on-orbit serving missions, such as object deorbits, spacecraft approaches, and automatic rendezvous and docking back-up systems. Teleoperation rendezvous and docking in lunar orbit may encounter bottlenecks for the inherent time delay in the communication link and the limited measurement accuracy of sensors. Moreover, human intervention is unsuitable in view of the partial communication coverage problem. To solve these problems, a shared control strategy for teleoperation rendezvous and docking is detailed. The control authority in lunar orbital maneuvers that involves two spacecraft as rendezvous and docking in the final phase was discussed in this paper. The predictive display model based on the relative dynamic equations is established to overcome the influence of the large time delay in communication link. We discuss and attempt to prove via consistent, ground-based simulations the relative merits of fully autonomous control mode (i.e., onboard computer-based), fully manual control (i.e., human-driven at the ground station) and shared control mode. The simulation experiments were conducted on the nine-degrees-of-freedom teleoperation rendezvous and docking simulation platform. Simulation results indicated that the shared control methods can overcome the influence of time delay effects. In addition, the docking success probability of shared control method was enhanced compared with automatic and manual modes.

A shared control method for obstacle avoidance with mobile robots and its interaction with communication delay

Teleoperation allows human operators to safely extend themselves to remote environments that are typically difficult or dangerous to access. The remote environments are often unstructured (i.e. not having clear roads or paths to follow) and only accessible by wireless communication (introducing factors such as degraded signals and communication delay). Teleoperated driving under these conditions can result in slow operation speeds and unintended collisions with obstacles. Automating portions of the teleoperation task can help mitigate some of the negative effects of wireless communication. Shared control is used to combine inputs from the human teleoperator and automation. This work presents a new model predictive control based shared control method. We introduce a new representation for obstacle free regions that works well with unstructured robot environments and allows for an model predictive control problem formulation that can be solved rapidly. The shared control method is implemented in a robot simulator and tested with human subjects. Two user studies involving a search task with a mobile robot evaluate the effectiveness of the shared control method and explore its interaction with factors such as communication delay and input interface style. Communication delay is found to have the largest magnitude effect on performance and safety measures. Results demonstrate that the shared control method can improve both performance and safety when delays are present.

Hierarchical Shared Control of Cane-Type Walking-Aid Robot.

A hierarchical shared-control method of the walking-aid robot for both human motion intention recognition and the obstacle emergency-avoidance method based on artificial potential field (APF) is proposed in this paper. The human motion intention is obtained from the interaction force measurements of the sensory system composed of 4 force-sensing registers (FSR) and a torque sensor. Meanwhile, a laser-range finder (LRF) forward is applied to detect the obstacles and try to guide the operator based on the repulsion force calculated by artificial potential field. An obstacle emergency-avoidance method which comprises different control strategies is also assumed according to the different states of obstacles or emergency cases. To ensure the user's safety, the hierarchical shared-control method combines the intention recognition method with the obstacle emergency-avoidance method based on the distance between the walking-aid robot and the obstacles. At last, experiments validate the effectiveness of the proposed hierarchical shared-control method.

Interactive Teleoperation of a Mobile Manipulator Using a Shared-Control Approach

This paper presents a shared-control-based design for teleoperation of a dual-arm omnidirectional mobile robot—a robot that can execute tasks commanded by a remotely user, or that can operate autonomously by adapting to its immediate environment, or that can operate in both modes simultaneously. To achieve this capability for the robot and for its effective remote control, a shared-control method has been developed. The controller determines the control gains for both the human and the autonomous commands by computing the user's confidence factor. For ease of use, a smartphone/tablet is used to control the robot. Using this approach, the robot allows the user to manipulate its motion from a remote site and compensates for a lack of local human knowledge. Practical experiments validate the proposed design and demonstrate the potential of the proposed shared-control method for home-service tasks.

Shared control of a medical robot with haptic guidance.

Tele-operation of robotic surgery reduces the radiation exposure during the interventional radiological operations. However, endoscope vision without force feedback on the surgical tool increases the difficulty for precise manipulation and the risk of tissue damage. The shared control of vision and force provides a novel approach of enhanced control with haptic guidance, which could lead to subtle dexterity and better maneuvrability during MIS surgery. The paper provides an innovative shared control method for robotic minimally invasive surgery system, in which vision and haptic feedback are incorporated to provide guidance cues to the clinician during surgery. The incremental potential field (IPF) method is utilized to generate a guidance path based on the anatomy of tissue and surgical tool interaction. Haptic guidance is provided at the master end to assist the clinician during tele-operative surgical robotic task. The approach has been validated with path following and virtual tumor targeting experiments. The experiment results demonstrate that comparing with vision only guidance, the shared control with vision and haptics improved the accuracy and efficiency of surgical robotic manipulation, where the tool-position error distance and execution time are reduced. The validation experiment demonstrates that the shared control approach could help the surgical robot system provide stable assistance and precise performance to execute the designated surgical task. The methodology could also be implemented with other surgical robot with different surgical tools and applications.

Shared Control of Teleoperation Rendezvous and Docking in Lunar Orbit

AbstractTeleoperation rendezvous and docking (RVD) in the lunar orbit may not reach the required RVD accuracy because of the inherent time delay in the communication link and the measurement error. To solve this problem, the shared control strategy for teleoperation RVD based on the potential field function was studied in this work. On the basis of the relative dynamic equations, a predictive display model to depict the predictive ability of the operator was established to overcome the time delay effect. The automatic mode was designed based on the potential field function, which regarded the relative distance and speed with respect to the safe corridor as reference quantities. In addition, this article outlines the concept of shared control weighting coefficient and its derivation on the basis of the influencing factors. Finally, a validation experiment was conducted on a nine-degree teleoperation RVD simulator. Results showed that the shared control method can overcome the influence of time delay in the...