Mind Control of a Robotic Arm With Visual Fusion Technology

Abstract

This paper reports the development of an intelligent shared control system for a robotic manipulator that is commanded by the user's mind. The target objects are detected by a vision system and then displayed to the user in a video that shows them fused with flicking diamonds that are designed to excite electroencephalograph (EEG) signals at different frequency bands. Through the analysis of the invoked EEG signals, a brain–computer interface is developed to infer the exact object that is required by the user. These results are then transferred to the shared control system, which is enabled by visual servoing techniques to achieve accurate object manipulation. The task motion and self-motion (CTS) methods are coordinated to enhance the intelligence of the shared control system by equipping the robot with an autonomous obstacle avoidance function. Extensive experimental studies are performed to verify that the adaptive object tracking algorithm, the CTS method, and the least-squares method are helpful in improving the performance of the intelligent robotic system.