Assessing internal, external and covert visuospatial attention in AR using brain sensing: a pilot study

Abstract

Most research works featuring AR and Brain-Computer Interface (BCI) systems are not taking advantage of the opportunities to integrate the two planes of data. In this project, we propose a prototype which combines an existing AR headset HoloLens 2 with a Brain-Computer Interfaces (BCI) system and we perform several tasks to validate this concept. In the first experiment, we reduced the distraction of the user by including information about the current attentional state. A simple game was designed for the Microsoft HoloLens 2, which changed in real time according to the user’s state of attention measured via electroencephalography (EEG). The system only responded if the attentional orientation was classified as "external." Fourteen users tested the attention-aware system; we show that the augmentation of the interface improved the usability of the system. We conclude that more systems would benefit from clearly visualizing the user’s ongoing attentional state as well as further efficient integration of AR and BCI headsets. In the second prototype we propose a system based on covert visuospatial attention (CVSA) – the process of focusing attention on different regions of the visual field without overt eye movements. In the experiment we did not rely on any stimulusdriven responses. Fourteen participants were able to test the system over the course of two days. To the best of our knowledge, this system is the first AR EEG-BCI integrated prototype that explores the complementary features of the AR headset like the HoloLens 2 and the CVSA paradigm.