A Multi-perception fusion using shared-control method for brain-mobile robot

This paper studies the man-machine shared control method based on man-machine mutual trust model and environmental information to enhance the working efficiency of coal mine rescue robots. Firstly, the framework of shared control system and working mode for the rescue robots are designed. Secondly, a mutual trust model between humans and machines is established by integrating the status of the operator, the performance of the robot, and the environmental information. Then, a human-machine shared control method is proposed, where the shared control coefficient is reasonably allocated by a fuzzy strategy taking human-machine mutual trust and environmental information as input. Finally, a robot experimental platform is used to illustrate the effectiveness of the proposed shared control method. The results show that the shared control method is of higher efficiency than teleoperation operation and autonomous control methods.

Teleoperation allows human operators to safely extend themselves to remote environments that are typically difficult or dangerous to access. The remote environments are often unstructured (i.e. not having clear roads or paths to follow) and only accessible by wireless communication (introducing factors such as degraded signals and communication delay). Teleoperated driving under these conditions can result in slow operation speeds and unintended collisions with obstacles. Automating portions of the teleoperation task can help mitigate some of the negative effects of wireless communication. Shared control is used to combine inputs from the human teleoperator and automation. This work presents a new model predictive control based shared control method. We introduce a new representation for obstacle free regions that works well with unstructured robot environments and allows for an model predictive control problem formulation that can be solved rapidly. The shared control method is implemented in a robot simulator and tested with human subjects. Two user studies involving a search task with a mobile robot evaluate the effectiveness of the shared control method and explore its interaction with factors such as communication delay and input interface style. Communication delay is found to have the largest magnitude effect on performance and safety measures. Results demonstrate that the shared control method can improve both performance and safety when delays are present.

In this paper, aiming at the problem of controlling conversion between driver and autopilot system, a shared control method based on priority weights is presented for controlling the lateral motion of intelligent vehicle. First, using the two-freedom vehicle dynamic model, this paper designs an autonomous driving controller using the LQR path tracking method. And then, a fuzzy controller based on driver's intention is designed. Next, an intelligent vehicle lateral controller based on priority weights is constructed according to the shared control method. Finally, through the co-simulation by MATLAB and CarSim, the simulation results show that the controller can track the planned path well, and the lateral controller based on shared control can keep the distance deviation within a small range.

Inspired by dual-user shared control method, a kind of auxiliary asymmetric shared control method for teleoperation is proposed in this paper. Firstly, we model the architecture for asymmetric shared control method, which contains three dominance factors with more flexibility and better transparency. In addition, we conclude some properties for asymmetric shared control architecture by defining some kinesthetic performance functions and present system stability conditions. Then we found a special dual-user operation scene—master 1 controls the slave robot alone and master 2 provides protection for master 1's operation. An auxiliary asymmetric shared control method is build considering the distance and velocity difference limits of the masters and adaptive dominance factors. In the end, two simulations are taken to certify the effectiveness of the proposed method. The simulation results show that though the force generated by protection actions influence the system transparency when the slave works in warning operation area, the auxiliary method has little influence to system kinesthetic performance.

Mobile robots can complete a task in cooperation with a human partner. In this letter, a hybrid shared control method for a mobile robot with omnidirectional wheels is proposed. A human partner utilizes a six degrees of freedom haptic device and electromyography (EMG) signals sensor to control the mobile robot. A hybrid shared control approach based on EMG and artificial potential field is exploited to avoid obstacles according to the repulsive force and attractive force and to enhance the human perception of the remote environment based on force feedback of the mobile platform. This shared control method enables the human partner to tele-control the mobile robot's motion and achieve obstacles avoidance synchronously. Compared with conventional shared control methods, this proposed one provides a force feedback based on muscle activation and drives the human partners to update their control intention with predictability. Experimental results demonstrate the enhanced performance of the mobile robots in comparison with the methods in the literature.

It is hard to grasp objects based on brain–computer interface (BCI) by brain-actuated robot arm and hand due to its high degree of freedom. Shared control strategy and hybrid BCI are research trends to solve this control problem of brain-actuated discrete event system. We propose a new shared control method based on fused fuzzy Petri nets (PNs) for combining the robot automatic control (AC) and the brain-actuated control. This method takes the advantages of both fuzzy control and PNs such as easy modeling, robustness, and effectiveness. Both MATLAB simulation test and Barrett robot hand practical experiments show that the proposed method performs much better than AC or BCI control independently. In the online BCI practical experiment, the user successfully control the Barrett robot hand to grasp object avoiding obstacle in whole ten random scenes by our shared control method and hybrid BCI. Compared with BCI control, the user needs not to synchronously work according to the specific paradigm in the whole process. Meanwhile, our method improves safety and robustness by comparing with AC.

In teleoperation tasks, communication delays between master and slave sides negatively affect the stability and transparency of closed loop system, and make it difficult to maintain a desired contact force. In order to improve the performance of contact force control in teleoperation under large time delays, a shared compliant control method is proposed in this paper. On the master side, the operator issues a desired contact force command according to the feedback of the contact stiffness identified on line besides motion commands. And on the slave side, a local contact force controller is designed using an adaptive Smith predictor, so as to shares control load with the operator. Experimental results show that this method can improve the force control performance, lower the difficulty of operation, and help the operator complete contact tasks with proper forces.In teleoperation tasks, communication delays between master and slave sides negatively affect the stability and transparency of closed loop system, and make it difficult to maintain a desired contact force. In order to improve the performance of contact force control in teleoperation under large time delays, a shared compliant control method is proposed in this paper. On the master side, the operator issues a desired contact force command according to the feedback of the contact stiffness identified on line besides motion commands. And on the slave side, a local contact force controller is designed using an adaptive Smith predictor, so as to shares control load with the operator. Experimental results show that this method can improve the force control performance, lower the difficulty of operation, and help the operator complete contact tasks with proper forces.

The development of an intelligent wheelchair (IW) platform that may be easily adapted to any commercial wheelchair and aid any person with special mobility needs is the main objective of this project. To be able to achieve this main objective, three distinct control methods were implemented in the IW: manual, shared and automatic. Several manual, shared and automatic control algorithms were developed for this task. This paper presents three of the most significant of those algorithms with emphasis on the shared control method. Experiments were performed, using a realistic simulator, with real users suffering from cerebral palsy in order to validate the approach. These experiments enabled to conclude which were the best shared control methods to implement on the IW. The experiments also revealed the importance of using shared (aided) controls for users with severe disabilities. The patients still felt having complete control over the wheelchair movement when using a shared control at a 50% level and thus this control type was very well accepted and should be used in intelligent wheelchairs since it is able to correct the direction in case of involuntary movements of the user but still gives him a sense of complete control over the IW movement.

This paper deals with the problem of needle steering in deformable tissues subject to physiological motions. A novel shared control method is proposed, which combines an automatic needle steering algorithm with the motions applied by the radiologist, in order to place the needle tip at the desired location. The core motivation is to leave potentially dangerous decisions and actions to the practitioner, whereas complex non-intuitive manipulations of the needle are performed automatically, in particular to compensate for breathing motions. The most original part of the present work lies in the method used to combine user inputs with a closed-loop automatic needle steering control method based on inverse Finite Element simulations. The method is evaluated with a realistic virtual environment using 2D X-ray projection images. The results are compared with those obtained with a fully teleoperated system, on the one hand, and with a fully automatic solution, on the other hand. These experiments show that the shared control solution allows for a better needle tip placement when only projection imaging is available.

The development of an intelligent wheel chair (IW) platform that may be easily adapted to any commercial electric powered wheelchair and aid any person with special mobility needs is the main objective of the IntellWheels project. To be able to achieve this main objective, three distinct control methods were implemented in the IW: manual, shared and automatic. Several algorithms were developed for each of these control methods. This paper presents three of the most significant of those algorithms with emphasis on the shared control method. Experiments were performed by users suffering from cerebral palsy, using a realistic simulator, in order to validate the approach. The experiments revealed the importance of using shared (aided) controls for users with severe disabilities. The patients still felt having complete control over the wheelchair movement when using a shared control at a 50 % level and thus this control type was very well accepted. Thus it may be used in intelligent wheelchairs since it is able to correct the direction in case of involuntary movements of the user but still gives him a sense of complete control over the IW movement.

Teleoperation rendezvous and docking (RVD) in the lunar orbit may not reach the required RVD accuracy because of the inherent time delay in the communication link and the measurement error. To solve this problem, the shared control strategy for teleoperation RVD based on the potential field function was studied in this work. On the basis of the relative dynamic equations, a predictive display model to depict the predictive ability of the operator was established to overcome the time delay effect. The automatic mode was designed based on the potential field function, which regarded the relative distance and speed with respect to the safe corridor as reference quantities. In addition, this article outlines the concept of shared control weighting coefficient and its derivation on the basis of the influencing factors. Finally, a validation experiment was conducted on a nine-degree teleoperation RVD simulator. Results showed that the shared control method can overcome the influence of time delay in the...

Performing a number of motion patterns — referred to as skills — (e.g., wave, spiral, sweeping motions) during teleoperation is an integral part of many industrial processes such as spraying, welding, and wiping (cleaning, polishing). Maintaining these motions whilst simultaneously avoiding obstacles and traversing complex terrain requires expert operators. In this work, we propose a novel skill-based shared control framework for incorporating the notion of skill assistance to aid novice operators to sustain these motion patterns whilst adhering to environmental constraints. Our shared control method uses streaming joystick data to estimate the model parameters that provide a description of the operator’s intention. We introduce a novel parametrization for state and control that combines skill and underlying trajectory models, leveraging a special type of curve known as Clothoids. This new parameterization allows for efficient computation of skill-based short term horizon plans, enabling the use of a Model Predictive Control (MPC) loop. We perform experiments on a hardware mock-up, validating the effectiveness of our method to recognize a switch of intended skill, and showing an improved quality of output motion, even under dynamically changing obstacles. See our accompanying video here: https://youtu.be/TwhsgA6fw6M.

This paper addresses the problem of safe and efficient navigation in remotely controlled robots operating in hazardous and unstructured environments; or conducting other remote robotic tasks. A shared control method is presented which blends the commands from a VFH+ obstacle avoidance navigation module with the teleoperation commands provided by an operator via a joypad. The presented approach offers several advantages such as flexibility allowing for a straightforward adaptation of the controller's behaviour and easy integration with variable autonomy systems; as well as the ability to cope with dynamic environments. The advantages of the presented controller are demonstrated by an experimental evaluation in a disaster response scenario. More specifically, presented evidence show a clear performance increase in terms of safety and task completion time compared to a pure teleoperation approach, as well as an ability to cope with previously unobserved obstacles.

Shared control is a common used method for human and wheelchair cooperation. However, most of the previous shared control methods didn't think much of the effect caused by the difference in the user's control ability. The control weight of a user in these methods is irrelevant to the user's capability or the driving conditions. In this paper, a dynamic shared control method is proposed to adapt wheelchair's assistance to the variations of user performance and the environmental changes. Three evaluation indices including safety, comfort and obedience are designed to evaluate wheelchair performance in real time. A minimax multi-objective optimization algorithm is adopted to calculate the user's control weight. The results of lab experiments and elderly home field tests show that this method can adapt the degree of wheelchair's autonomy to the user's control ability and it makes driving wheelchair much easier for elder people.

Shared control on lunar spacecraft teleoperation rendezvous operations with large time delay