Deep reinforcement learning from error-related potentials via an EEG-based brain-computer interface

Abstract

Reinforcement learning (RL) from human preferences suffered from temporal and interaction environments’ limitations, which rule out real-time and real-world robotic applications of deep RL. To overcome the limitations, this study introduced the electroencephalography (EEG)-measured error-related potentials (ErrPs) to train a robotic RL system based on a brain-computer interface (BCI). We decoded ErrP signals by selecting human preferences in real-time to train robotic behavior by deep RL during a binary object selection task. Twelve healthy subjects participated in the ErrP experiments, in which they were asked to select and adjust self-favored behavior after a machine’s random selections. The decoded ErrP signals classified by a convolutional neural network (CNN)architecture to achieve an average classification accuracy and an area under the ROC curve of 67.49% and 0.639, respectively. By using the well-trained ErrP signals classifier to train the deep RL system, our final results for training robotic behavior through ErrP-based preferences showed an average of 15.21% improvement in efficiency while obtaining acceptable rewards in RL. Thus, the work used brain signals instead of pressing or clicking buttons as the rewards of RL, and constructed a real-time and free from interaction interference intuitive RL system.