Brain-computer Interface Community Research Articles

Building consensus on clinical outcome assessments for BCI devices. A summary of the 10th BCI society meeting 2023 workshop.

The 10th International Brain Computer Interface (BCI) Society Meeting, 'Balancing Innovation and Translation', was held from the 6th to 9th of June 2023 in Brussels, Belgium. This report provides a summary of the workshop 'Building Consensus on Clinical Outcome Assessments (COAs) for BCI Devices'. This workshop was intended to give participants an overview of the current state of BCI, future opportunities, and how different countries and regions provide regulatory oversight to support the BCI community to develop safe and effective devices for patients. Five presentations and a panel discussion including representatives from regulators, industry, and clinical research stakeholders focused on how various stakeholders and the BCI community might best work together to ensure studies provide data that is useful for evaluating safety and effectiveness, including reaching consensus on clinical outcome assessments (COAs) that represent clinically meaningful benefits and support regulatory and payor requirements. This report focuses on the regulatory and reimbursement requirements for medical devices and how to best measure safety and effectiveness and summarizes the presentations from five experts and the discussion between the panel and the audience. Consensus was reached on the following items specifically related to BCI: (i) the importance of and need for a new generation of COAs, (ii) the challenges facing the development of appropriate clinical outcome assessments, and (iii) that improvements in COAs should demonstrate obvious and clinically meaningful benefit(s). There was discussion on: (i) clinical trial design for BCIs and (ii) considerations for payor reimbursement and other funding. Whilst the importance of building community consensus on COAs was apparent, further collaboration will be required to reach consensus on which specific current and/or novel COAs could be used for the BCI field to evolve from research to market.

Pseudo-online framework for BCI evaluation: a MOABB perspective using various MI and SSVEP datasets

Objective. BCI (Brain–Computer Interfaces) operate in three modes: online, offline, and pseudo-online. In online mode, real-time EEG data is constantly analyzed. In offline mode, the signal is acquired and processed afterwards. The pseudo-online mode processes collected data as if they were received in real-time. The main difference is that the offline mode often analyzes the whole data, while the online and pseudo-online modes only analyze data in short time windows. Offline processing tends to be more accurate, while online analysis is better for therapeutic applications. Pseudo-online implementation approximates online processing without real-time constraints. Many BCI studies being offline introduce biases compared to real-life scenarios, impacting classification algorithm performance. Approach. The objective of this research paper is therefore to extend the current MOABB framework, operating in offline mode, so as to allow a comparison of different algorithms in a pseudo-online setting with the use of a technology based on overlapping sliding windows. To do this will require the introduction of a idle state event in the dataset that takes into account all different possibilities that are not task thinking. To validate the performance of the algorithms we will use the normalized Matthews correlation coefficient and the information transfer rate. Main results. We analyzed the state-of-the-art algorithms of the last 15 years over several motor imagery and steady state visually evoked potential multi-subjects datasets, showing the differences between the two approaches from a statistical point of view. Significance. The ability to analyze the performance of different algorithms in offline and pseudo-online modes will allow the BCI community to obtain more accurate and comprehensive reports regarding the performance of classification algorithms.

MetaBCI: An open-source platform for brain–computer interfaces

Background:Recently, brain–computer interfaces (BCIs) have attracted worldwide attention for their great potential in clinical and real-life applications. To implement a complete BCI system, one must set up several links to translate the brain intent into computer commands. However, there is not an open-source software platform that can cover all links of the BCI chain. Method:This study developed a one-stop open-source BCI software, namely MetaBCI, to facilitate the construction of a BCI system. MetaBCI is written in Python, and has the functions of stimulus presentation (Brainstim), data loading and processing (Brainda), and online information flow (Brainflow). This paper introduces the detailed information of MetaBCI and presents four typical application cases. Results:The results showed that MetaBCI was an extensible and feature-rich software platform for BCI research and application, which could effectively encode, decode, and feedback brain activities. Conclusions:MetaBCI can greatly lower the BCI’s technical threshold for BCI beginners and can save time and cost to build up a practical BCI system. The source code is available at https://github.com/TBC-TJU/MetaBCI, expecting new contributions from the BCI community.

Real-Time Navigation in Google Street View® Using a Motor Imagery-Based BCI.

Navigation in virtual worlds is ubiquitous in games and other virtual reality (VR) applications and mainly relies on external controllers. As brain-computer interfaces (BCI)s rely on mental control, bypassing traditional neural pathways, they provide to paralyzed users an alternative way to navigate. However, the majority of BCI-based navigation studies adopt cue-based visual paradigms, and the evoked brain responses are encoded into navigation commands. Although robust and accurate, these paradigms are less intuitive and comfortable for navigation compared to imagining limb movements (motor imagery, MI). However, decoding motor imagery from EEG activity is notoriously challenging. Typically, wet electrodes are used to improve EEG signal quality, including a large number of them to discriminate between movements of different limbs, and a cuedbased paradigm is used instead of a self-paced one to maximize decoding performance. Motor BCI applications primarily focus on typing applications or on navigating a wheelchair-the latter raises safety concerns-thereby calling for sensors scanning the environment for obstacles and potentially hazardous scenarios. With the help of new technologies such as virtual reality (VR), vivid graphics can be rendered, providing the user with a safe and immersive experience; and they could be used for navigation purposes, a topic that has yet to be fully explored in the BCI community. In this study, we propose a novel MI-BCI application based on an 8-dry-electrode EEG setup, with which users can explore and navigate in Google Street View®. We pay attention to system design to address the lower performance of the MI decoder due to the dry electrodes' lower signal quality and the small number of electrodes. Specifically, we restricted the number of navigation commands by using a novel middle-level control scheme and avoided decoder mistakes by introducing eye blinks as a control signal in different navigation stages. Both offline and online experiments were conducted with 20 healthy subjects. The results showed acceptable performance, even given the limitations of the EEG set-up, which we attribute to the design of the BCI application. The study suggests the use of MI-BCI in future games and VR applications for consumers and patients temporarily or permanently devoid of muscle control.

Fixed template network and dynamic template network: novel network designs for decoding steady-state visual evoked potentials

Objective. Decomposition methods are efficient to decode steady-state visual evoked potentials (SSVEPs). In recent years, the brain–computer interface community has also been developing deep learning networks for decoding SSVEPs. However, there is no clear evidence that current deep learning models outperform decomposition methods on the SSVEP decoding tasks. Many studies lacked the comparison with state-of-the-art decomposition methods in a fair environment. Approach. This study proposed a novel network design motivated by the works of decomposition methods. Fixed template network (FTN) and dynamic template network (DTN) are two novel networks combining the advantages of fixed templates and subject-specific templates. This study also proposed a data augmentation method for SSVEPs. This study compared the intra-subject classification performance of DTN and FTN with that of state-of-the-art decomposition methods on three public SSVEP datasets. Main results. The results show that both FTN and DTN achieved the suboptimal classification performance compared with state-of-the-art decomposition methods. Significance. Both network designs could enhance the decoding performance of SSVEPs, making them promising networks for improving the practicality of SSVEP-based applications.

Methodological Recommendations for Studies on the Daily Life Implementation of Implantable Communication-Brain–Computer Interfaces for Individuals With Locked-in Syndrome

Implantable brain–computer interfaces (BCIs) promise to be a viable means to restore communication in individuals with locked-in syndrome (LIS). In 2016, we presented the world-first fully implantable BCI system that uses subdural electrocorticography electrodes to record brain signals and a subcutaneous amplifier to transmit the signals to the outside world, and that enabled an individual with LIS to communicate via a tablet computer by selecting icons in spelling software. For future clinical implementation of implantable communication-BCIs, however, much work is still needed, for example, to validate these systems in daily life settings with more participants, and to improve the speed of communication. We believe the design and execution of future studies on these and other topics may benefit from the experience we have gained. Therefore, based on relevant literature and our own experiences, we here provide an overview of procedures, as well as recommendations, for recruitment, screening, inclusion, imaging, hospital admission, implantation, training, and support of participants with LIS, for studies on daily life implementation of implantable communication-BCIs. With this article, we not only aim to inform the BCI community about important topics of concern, but also hope to contribute to improved methodological standardization of implantable BCI research.

Multi-Armed Bandits in Brain-Computer Interfaces.

The multi-armed bandit (MAB) problem models a decision-maker that optimizes its actions based on current and acquired new knowledge to maximize its reward. This type of online decision is prominent in many procedures of Brain-Computer Interfaces (BCIs) and MAB has previously been used to investigate, e.g., what mental commands to use to optimize BCI performance. However, MAB optimization in the context of BCI is still relatively unexplored, even though it has the potential to improve BCI performance during both calibration and real-time implementation. Therefore, this review aims to further describe the fruitful area of MABs to the BCI community. The review includes a background on MAB problems and standard solution methods, and interpretations related to BCI systems. Moreover, it includes state-of-the-art concepts of MAB in BCI and suggestions for future research.

Current Challenges for the Practical Application of Electroencephalography-Based Brain–Computer Interfaces

Abstract Although recent brain-computer interface (BCI) studies have achieved tremendous progress in increasing communication commands, measuring the level of sub-microvolt of EEG amplitude, and so on, it is still challenging to make the leap from the lab to the marketplace, which hampers BCI applicability. This article highlights two formidable challenges that the BCI community should pay more attention to. Then we further analyze the reasons and summarize several important research topics that are expected to overcome these challenges. We hope these topics could lead to more discussions and studies in the BCI community.

A Systematic Review on Motor-Imagery Brain-Connectivity-Based Computer Interfaces

This review article discusses the definition and implementation of brain–computer interface (BCI) system relying on brain connectivity (BC) and machine learning/deep learning (DL) for motor imagery (MI)-based applications. During the past few years, many approaches have been explored in terms of types of neurological sources of information, feature extraction, and intention prediction for BCI applications. Two novel aspects are becoming increasingly interesting for the BCI community: BC modeling and DL. The former aims at describing the interactions among different brain regions as connectivity patterns that reflect the dynamics of information flow either at rest or when performing a task. The latter is becoming pervasive for its capability of modeling and predicting complex data, where a huge amount of information is involved. In this scenario, we conducted a systematic literature review on BCI studies that led to the selection of 34 articles meeting all the required criteria. This provides evidence of the rapid growth of the topic over the past few years, though being still in its infancy. The last part of this article is dedicated to this new frontier of BCI that we call MI BC-based computer interfaces highlighting the potential of BC features. This, jointly with DL as enabling technology, has the potential of improving the performance of electroencephalography-based systems.

Imagined speech increases the hemodynamic response and functional connectivity of the dorsal motor cortex

Objective. Decoding imagined speech from brain signals could provide a more natural, user-friendly way for developing the next generation of the brain–computer interface (BCI). With the advantages of non-invasive, portable, relatively high spatial resolution and insensitivity to motion artifacts, the functional near-infrared spectroscopy (fNIRS) shows great potential for developing the non-invasive speech BCI. However, there is a lack of fNIRS evidence in uncovering the neural mechanism of imagined speech. Our goal is to investigate the specific brain regions and the corresponding cortico-cortical functional connectivity features during imagined speech with fNIRS. Approach. fNIRS signals were recorded from 13 subjects’ bilateral motor and prefrontal cortex during overtly and covertly repeating words. Cortical activation was determined through the mean oxygen–hemoglobin concentration changes, and functional connectivity was calculated by Pearson’s correlation coefficient. Main results. (a) The bilateral dorsal motor cortex was significantly activated during the covert speech, whereas the bilateral ventral motor cortex was significantly activated during the overt speech. (b) As a subregion of the motor cortex, sensorimotor cortex (SMC) showed a dominant dorsal response to covert speech condition, whereas a dominant ventral response to overt speech condition. (c) Broca’s area was deactivated during the covert speech but activated during the overt speech. (d) Compared to overt speech, dorsal SMC(dSMC)-related functional connections were enhanced during the covert speech. Significance. We provide fNIRS evidence for the involvement of dSMC in speech imagery. dSMC is the speech imagery network’s key hub and is probably involved in the sensorimotor information processing during the covert speech. This study could inspire the BCI community to focus on the potential contribution of dSMC during speech imagery.

Semi-Supervised Learning for Auditory Event-Related Potential-Based Brain–Computer Interface

A brain-computer interface (BCI) is a communication tool that analyzes neural activity and relays the translated commands to carry out actions. In recent years, semi-supervised learning (SSL) has attracted attention for visual event-related potential (ERP)-based BCIs and motor-imagery BCIs as an effective technique that can adapt to the variations in patterns among subjects and trials. The applications of the SSL techniques are expected to improve the performance of auditory ERP-based BCIs as well. However, there is no conclusive evidence supporting the positive effect of SSL techniques on auditory ERP-based BCIs. If the positive effect could be verified, it will be helpful for the BCI community. In this study, we assessed the effects of SSL techniques on two public auditory BCI datasets-AMUSE and PASS2D-using the following machine learning algorithms: step-wise linear discriminant analysis, shrinkage linear discriminant analysis, spatial temporal discriminant analysis, and least-squares support vector machine. These backbone classifiers were firstly trained by labeled data and incrementally updated by unlabeled data in every trial of testing data based on SSL approach. Although a few data of the datasets were negatively affected, most data were apparently improved by SSL in all cases. The overall accuracy was logarithmically increased with every additional unlabeled data. This study supports the positive effect of SSL techniques and encourages future researchers to apply them to auditory ERP-based BCIs.

A feasibility study of a complete low-cost consumer-grade brain-computer interface system.

Brain-computer interfaces (BCIs) are technologies that provide the user with an alternative way of communication. A BCI measures brain activity (e.g. EEG) and converts it into output commands. Motor imagery (MI), the mental simulation of movements, can be used as a BCI paradigm, where the movement intention of the user can be translated into a real movement, helping patients in motor recovery rehabilitation. One of the main limitations for the broad use of such devices is the high cost associated with the high-quality equipment used for capturing the biomedical signals. Different low-cost consumer-grade alternatives have emerged with the objective of bringing these systems closer to the final users. The quality of the signals obtained with such equipments has already been evaluated and found to be competitive with those obtained with well-known clinical-grade devices. However, how these consumer-grade technologies can be integrated and used for practical MI-BCIs has not yet been explored. In this work, we provide a detailed description of the advantages and disadvantages of using OpenBCI boards, low-cost sensors and open-source software for constructing an entirely consumer-grade MI-BCI system. An analysis of the quality of the signals acquired and the MI detection ability is performed. Even though communication between the computer and the OpenBCI board is not always stable and the signal quality is sometimes affected by ambient noise, we find that by means of a filter-bank based method, similar classification performances can be achieved with an MI-BCI built under low-cost consumer-grade devices as compared to when clinical-grade systems are used. By means of this work we share with the BCI community our experience on working with emerging low-cost technologies, providing evidence that an entirely low-cost MI-BCI can be built. We believe that if communication stability and artifact rejection are improved, these technologies will become a valuable alternative to clinical-grade devices.

A Data Augmentation Scheme for Geometric Deep Learning in Personalized Brain–Computer Interfaces

Electroencephalography signals inherently deviate from the notion of regular spatial sampling, as they reflect the coordinated action from multiple distributed overlapping cortical networks. Hence, the observed brain dynamics are influenced both by the topology of the sensor array and the underlying functional connectivity. Neural engineers are currently exploiting the advances in the domain of graph signal processing in an attempt to create robust and reliable brain decoding systems. In this direction, Geometric Deep Learning is a highly promising concept for combining the benefits of graph signal processing and deep learning towards revolutionising Brain-Computer Interfaces (BCIs). However, its exploitation has been hindered by its data-demanding character. As a remedy, we propose here a novel data augmentation approach that combines the multiplex network modelling of multichannel signal with a graph variant of the classical Empirical Mode Decomposition (EMD), and which proves to be a strong asset when combined with Graph Convolutional Neural Networks (GCNNs). As our graph-EMD algorithm makes no assumptions with respect to linearity and stationarity, it appears as an appealing solution towards analysing brain signals without artificially imposing regularities in either temporal or spatial domain. Our experimental results indicate that the proposed scheme for data augmentation leads to substantial improvement when it is combined with GCNNs. Using recordings from two distinct BCI applications and comparing against a state-of-the-art augmentation method, we illustrate the benefits from its use. By making it available to BCI community, we hope to further foster the application of geometric deep learning in the field.

Two Player Online Brain-Controlled Chess.

Brain-computer interfaces (BCIs) allow for translating brain signals into control commands, e.g. to control games. One of the biggest quests of the BCI community is to realize new exciting applications. In this paper, we present a two player online chess application where both players control their pieces solely with their brain activity. Control is realized in a two-step process where players first select the desired chess piece and then the field they want to move it to. A selection is accomplished with visual event-related potentials that are elicited by highlighting each possible piece or field one by one. Six healthy volunteers participated in our study by playing a game against each other in pairs over a free chess server. They successfully completed at least one match per pair. Our results show that even BCI-naive players obtain almost perfect control over the application. On average, 96% of the moves were correct. Most of the errors came from technical problems that can be corrected in future versions of the application. Given the high popularity of chess, this intuitive BCI game is an appealing new application for patients and for healthy users that want to explore BCIs.

Gumpy: a Python toolbox suitable for hybrid brain–computer interfaces

Objective. The objective of this work is to present gumpy, a new free and open source Python toolbox designed for hybrid brain–computer interface (BCI). Approach. Gumpy provides state-of-the-art algorithms and includes a rich selection of signal processing methods that have been employed by the BCI community over the last 20 years. In addition, a wide range of classification methods that span from classical machine learning algorithms to deep neural network models are provided. Gumpy can be used for both EEG and EMG biosignal analysis, visualization, real-time streaming and decoding. Results. The usage of the toolbox was demonstrated through two different offline example studies, namely movement prediction from EEG motor imagery, and the decoding of natural grasp movements with the applied finger forces from surface EMG (sEMG) signals. Additionally, gumpy was used for real-time control of a robot arm using steady-state visually evoked potentials (SSVEP) as well as for real-time prosthetic hand control using sEMG. Overall, obtained results with the gumpy toolbox are comparable or better than previously reported results on the same datasets. Significance. Gumpy is a free and open source software, which allows end-users to perform online hybrid BCIs and provides different techniques for processing and decoding of EEG and EMG signals. More importantly, the achieved results reveal that gumpy’s deep learning toolbox can match or outperform the state-of-the-art in terms of accuracy. This can therefore enable BCI researchers to develop more robust decoding algorithms using novel techniques and hence chart a route ahead for new BCI improvements.

MOABB: trustworthy algorithm benchmarking for BCIs

Objective. Brain–computer interface (BCI) algorithm development has long been hampered by two major issues: small sample sets and a lack of reproducibility. We offer a solution to both of these problems via a software suite that streamlines both the issues of finding and preprocessing data in a reliable manner, as well as that of using a consistent interface for machine learning methods. Approach. By building on recent advances in software for signal analysis implemented in the MNE toolkit, and the unified framework for machine learning offered by the scikit-learn project, we offer a system that can improve BCI algorithm development. This system is fully open-source under the BSD licence and available at https://github.com/NeuroTechX/moabb. Main results. We analyze a set of state-of-the-art decoding algorithms across 12 open access datasets, including over 250 subjects. Our results show that even for the best methods, there are datasets which do not show significant improvements, and further that many previously validated methods do not generalize well outside the datasets they were tested on. Significance. Our analysis confirms that BCI algorithms validated on single datasets are not representative, highlighting the need for more robust validation in the machine learning for BCIs community.

Optimal Control Signal for an EEG Based Casual BCI

BCI (Brain computer interface) is a control and communication system which allows electrophysiological activity to control a computer or a peripheral device directly, without taking the natural route of peripheral nerves and muscles. The prime motive behind developing BCI technology was its ability to act as the only interactive link for people disabled by amyotrophic lateral sclerosis (ALS), cerebral palsy, spinal cord injury, stroke and similar neuromuscular disorders of high severity. However in the last decade, a gradual shift in BCI end-users from patients to casual (healthy) individuals has increased significantly. Because of this shift, BCI community has recognized the need for EEG based casual BCI to be more efficient and user friendly, keeping in mind the customized needs of healthy (Casual) user. So for increasing the performance of such BCIs, the selection of optimal control signal plays a very significant role. Hence, in this work, we evaluate various EEG control signals (CS) in accordance with considerations relevant to user-friendliness of casual BCIs and point up their neuro-physiological origins as well as their effectiveness in current applications. Finally, we recommend a set of parameters for selection of optimal EEG based control signal for casual BCIs and the best suitable option available among the present day control signals.

A New Motor Imagery EEG Classification Method FB-TRCSP+RF Based on CSP and Random Forest

There is a general agreement in the brain computer interface community that the feature extracting method called the common spatial pattern (CSP) combined with nonlinear classifiers can provide excellent results in some cases. However, CSP is also known to be very sensitive to noise and prone to over fitting, and the performance of this spatial filter is closely related to the operational frequency band of electroencephalogram data. To address this issue, we propose a new method FB-TRCSP+RF based on CSP and random forest. The FB-TRCSP is combined by the 8th-order Butterworth bandpass-filters and the CSP with Tikhonov regularization, which is a more robust feature extraction method compared to the CSP. Then, the model is applied to an experimental data set collected from 14 subjects and is compared with the non-regularization method FB-CSP+RF. The results show that the method we proposed yields relatively higher median classification accuracies and shows a stronger ability in subject-to-subject learning compared to prevailing approaches.

Can graph metrics be used for EEG-BCIs based on hand motor imagery?

Abstract The study of motor imagery (MI) has been a subject of great interest within the brain-computer interface (BCI) community. Several approaches have been proposed to solve the problem of classifying cerebral responses due to MI, mostly based on the power spectral density of the mu and beta bands; however, no optimum manner of proceeding through the fundamental steps of a MI-BCI has yet been established. In this work, we explored a relatively novel approach regarding feature generation for a MI-BCI by assuming that functional connectivity patterns of the brain are altered during hand MI. We modelled interactions among EEG electrodes by a graph, extracted metrics from it during left and right hand MI from eight subjects and classified the signals using commonly employed techniques in the BCI community (LDA and SVM). We also compared this approach to the more established method of using the signal power spectral density as the classifier features. With the graph method, we confirmed that only specific electrodes provide relevant information for data classification. A first approach provided maximum average classification rates across all subjects for the graph method of 86% for the mu band and 87% for the beta band. For the PSD method, average rates were of 98% and 99% for the mu and beta bands, respectively. However, a much larger number of features was needed: (130 ± 44) and (273 ± 89) for the mu and beta bands, respectively. Aiming to reproduce these rates using the graph method, pairwise inputs combinations of graph metrics were tested. They proved to be sufficient to obtain essentially the same classification accuracy rates, but with a considerably smaller number of features – about 60 features, for both bands. We thus conclude that the graph method is a feasible option for classification of hand MI signals.

Learning from label proportions in brain-computer interfaces: Online unsupervised learning with guarantees.

ObjectiveUsing traditional approaches, a brain-computer interface (BCI) requires the collection of calibration data for new subjects prior to online use. Calibration time can be reduced or eliminated e.g., by subject-to-subject transfer of a pre-trained classifier or unsupervised adaptive classification methods which learn from scratch and adapt over time. While such heuristics work well in practice, none of them can provide theoretical guarantees. Our objective is to modify an event-related potential (ERP) paradigm to work in unison with the machine learning decoder, and thus to achieve a reliable unsupervised calibrationless decoding with a guarantee to recover the true class means.MethodWe introduce learning from label proportions (LLP) to the BCI community as a new unsupervised, and easy-to-implement classification approach for ERP-based BCIs. The LLP estimates the mean target and non-target responses based on known proportions of these two classes in different groups of the data. We present a visual ERP speller to meet the requirements of LLP. For evaluation, we ran simulations on artificially created data sets and conducted an online BCI study with 13 subjects performing a copy-spelling task.ResultsTheoretical considerations show that LLP is guaranteed to minimize the loss function similar to a corresponding supervised classifier. LLP performed well in simulations and in the online application, where 84.5% of characters were spelled correctly on average without prior calibration.SignificanceThe continuously adapting LLP classifier is the first unsupervised decoder for ERP BCIs guaranteed to find the optimal decoder. This makes it an ideal solution to avoid tedious calibration sessions. Additionally, LLP works on complementary principles compared to existing unsupervised methods, opening the door for their further enhancement when combined with LLP.