An online SSVEP-BCI system in an optical see-through augmented reality environment

Abstract

Objective. This study aimed to design and evaluate a high-speed online steady-state visually evoked potential (SSVEP)-based brain–computer interface (BCI) in an optical see-through (OST) augmented reality (AR) environment. Approach. An eight-class BCI was designed in an OST-AR headset which is wearable and allows users to see the user interface of the BCI and the device to be controlled in the same view field via the OST head-mounted display. The accuracies, information transfer rates (ITRs), and SSVEP signal characteristics of the AR-BCI were evaluated and compared with a computer screen-based BCI implemented with a laptop in offline and online cue-guided tasks. Then, the performance of the AR-BCI was evaluated in an online robotic arm control task. Main results. The offline results obtained during the cue-guided task performed with the AR-BCI showed maximum averaged ITRs of 65.50 ± 9.86 bits min−1 according to the extended canonical correlation analysis-based target identification method. The online cue-guided task achieved averaged ITRs of 65.03 ± 11.40 bits min−1. The online robotic arm control task achieved averaged ITRs of 45.57 ± 7.40 bits min−1. Compared with the screen-based BCI, some limitations of the AR environment impaired BCI performance and the quality of SSVEP signals. Significance. The results showed the potential for providing a high-performance brain–control interaction method by combining AR and BCI. This study could provide methodological guidelines for developing more wearable BCIs in OST-AR environments and will also encourage more interesting applications involving BCIs and AR techniques.