Hybrid brain-computer interface with motor imagery and error-related brain activity

Mahta Mousavi,Virginia R De Sa,Laurens R Krol

doi:10.1088/1741-2552/abaa9d

Abstract

Objective. Brain-computer interface (BCI) systems read and interpret brain activity directly from the brain. They can provide a means of communication or locomotion for patients suffering from neurodegenerative diseases or stroke. However, non-stationarity of brain activity limits the reliable transfer of the algorithms that were trained during a calibration session to real-time BCI control. One source of non-stationarity is the user’s brain response to the BCI output (feedback), for instance, whether the BCI feedback is perceived as an error by the user or not. By taking such sources of non-stationarity into account, the reliability of the BCI can be improved. Approach. In this work, we demonstrate a real-time implementation of a hybrid motor imagery BCI combining the information from the motor imagery signal and the error-related brain activity simultaneously so as to gain benefit from both sources. Main results. We show significantly improved performance in real-time BCI control across 12 participants, compared to a conventional motor imagery BCI. The significant improvement is in terms of classification accuracy, target hit rate, subjective perception of control and information-transfer rate. Moreover, our offline analyses of the recorded EEG data show that the error-related brain activity provides a more reliable source of information than the motor imagery signal. Significance. This work shows, for the first time, that the error-related brain activity classifier compared to the motor imagery classifier is more consistent when trained on calibration data and tested during online control. This likely explains why the proposed hybrid BCI allows for a more reliable means of communication or rehabilitation for patients in need.

Highlights

Since the introduction of a brain-computer interface (BCI) by Vidal [1], there have been many implementations as potential communication or rehabilitation interventions for patients (e.g. [2,3,4,5,6])
Since real-time feedback can itself modify brain activity, it is vital to evaluate the efficacy of our earlier proposed hybrid BCI in real-time control. This is the goal of the current study where we answer two questions: 1) how the performance of the proposed hybrid BCI in real-time control compares to a conventional motor imagery BCI, and 2) how the underlying two aspects of the proposed hybrid BCI compare with each other in their contributions to robust and reliable control
Scores were averaged across the right/left hand motor imagery (R/L) and R/L+good/bad classifier (G/B) blocks separately and reported as hit rates (HR) and subjective rate (SR) for

Summary

Introduction

Since the introduction of a brain-computer interface (BCI) by Vidal [1], there have been many implementations as potential communication or rehabilitation interventions for patients (e.g. [2,3,4,5,6]). To improve reliability in BCI control while avoiding time-consuming re-calibration sessions, one approach is to use other available sources of information in order to support, adjust, or correct the information from the primary detected signal One potential such source is the brain activity that occurs in response to the BCI output (feedback). A third approach based on error integration proposed a hybrid BCI for a 1-D cursor control by combining the motor imagery signal with the user brain activity in response to the cursor’s changes in the direction of movement [22, 23]. The proposed hybrid BCI system translated the classification score from the domain of the error-related brain activity to that of the motor imagery classifier and learned a logistic regression classifier to best combine the two sources of information for each user. This is the goal of the current study where we answer two questions: 1) how the performance of the proposed hybrid BCI in real-time control compares to a conventional motor imagery BCI, and 2) how the underlying two aspects of the proposed hybrid BCI (i.e. error-related brain activity and motor imagery signal) compare with each other in their contributions to robust and reliable control

Methods

Results

Discussion

Conclusion and future work