Brain-Controlled AR Feedback Design for User's Training in Surgical HRI

Abstract

Brain-computer interfaces (BCIs) offer high potential for enhancing training in many tasks, especially those that require maintaining high levels of concentration such as surgery. Training focus and attention can play a critical role in surgery since concentration on the task at hand is fundamental to prevent life-threatening errors. In this paper we propose a new method for concentration training in the context of robot-assisted laser microsurgery associated to a feedback design that makes the interaction more intuitive. This approach couples augmented reality (AR) features to both BCI-based on-line measurement of the user's mental focus and the control of the surgical robot. The methodology is described as a brain-controlled augmented reality (BcAR) training system. AR is used to maintain the surgeon's perceptual contact with the real operating setting, while focus stimulation is provided by modifying features of an AR item based on real-time monitoring of the user's mental state. In this research a low-cost EEG device is used and the BcAR is implemented in the form of an AR scalpel that behaves as a retractable knife according to the user's mental focus: low concentration levels retract the knife and prevent cutting. This design provides directional compatibility between the AR feedback animation and the spontaneous motion of user's attention along the AR tool, resulting in an intuitive system with real impact on the training outcome. This is demonstrated through user trials and comparison with training based on simple AR feedback (no EEG). Results demonstrate the potential of the approach, showing a significant improvement in post-training task execution time without any detriment to user experience. Subjective questionnaires also confirmed the critical role of directional compatibility in the AR feedback. Such findings allow the identification of further improvements and novel potential applications of this interaction paradigm.