Shared control of a robotic arm using non-invasive brain–computer interface and computer vision guidance

Abstract

Control of a robotic arm using a brain–computer interface (BCI) for reach and grasp activities is one of the most fascinating applications for some severely disabled people, which is especially challenging for the non-invasive BCIs based on electroencephalography (EEG). In this paper, shared control is applied to realize the control of a dexterous robotic arm with a motor imagery-based (MI-based) BCI and computer vision guidance. With the utilization of the shared control, the subjects just need to move the robotic arm by performing only two different mental tasks to the surrounding area of the target. The accurate pose of the target is estimated by a depth camera equipped in the robot system. Once the endpoint of the robotic arm enters the pre-defined vision-guided region, the robotic arm will grasp the target autonomously. Five healthy and inexperienced subjects participated in the online experiments and the average success rate is above 70% even with no specific user training. The results show that the shared control can make the robotic arm accomplish the complex tasks (reach and grasp) with the simple two-class MI-based BCIs.