Brain-computer Interface Users Research Articles

A Low-Complexity Brain-Computer Interface for High-Complexity Robot Swarm Control.

A brain-computer interface (BCI) is a system that allows a human operator to use only mental commands in controlling end effectors that interact with the world around them. Such a system consists of a measurement device to record the human user's brain activity, which is then processed into commands that drive a system end effector. BCIs involve either invasive measurements which allow for high-complexity control but are generally infeasible, or noninvasive measurements which offer lower quality signals but are more practical to use. In general, BCI systems have not been developed that efficiently, robustly, and scalably perform high-complexity control while retaining the practicality of noninvasive measurements. Here we leverage recent results from feedback information theory to fill this gap by modeling BCIs as a communications system and deploying a human-implementable interaction algorithm for noninvasive control of a high-complexity robot swarm. We construct a scalable dictionary of robotic behaviors that can be searched simply and efficiently by a BCI user, as we demonstrate through a large-scale user study testing the feasibility of our interaction algorithm, a user test of the full BCI system on (virtual and real) robot swarms, and simulations that verify our results against theoretical models. Our results provide a proof of concept for how a large class of high-complexity effectors (even beyond robotics) can be effectively controlled by a BCI system with low-complexity and noisy inputs.

A new error-monitoring brain–computer interface based on reinforcement learning for people with autism spectrum disorders

Objective. Brain–computer interfaces (BCIs) are emerging as promising cognitive training tools in neurodevelopmental disorders, as they combine the advantages of traditional computerized interventions with real-time tailored feedback. We propose a gamified BCI based on non-volitional neurofeedback for cognitive training, aiming at reaching a neurorehabilitation tool for application in autism spectrum disorders (ASDs). Approach. The BCI consists of an emotional facial expression paradigm controlled by an intelligent agent that makes correct and wrong actions, while the user observes and judges the agent’s actions. The agent learns through reinforcement learning (RL) an optimal strategy if the participant generates error-related potentials (ErrPs) upon incorrect agent actions. We hypothesize that this training approach will allow not only the agent to learn but also the BCI user, by participating through implicit error scrutiny in the process of learning through operant conditioning, making it of particular interest for disorders where error monitoring processes are altered/compromised such as in ASD. In this paper, the main goal is to validate the whole methodological BCI approach and assess whether it is feasible enough to move on to clinical experiments. A control group of ten neurotypical participants and one participant with ASD tested the proposed BCI approach. Main results. We achieved an online balanced-accuracy in ErrPs detection of 81.6% and 77.1%, respectively for two different game modes. Additionally, all participants achieved an optimal RL strategy for the agent at least in one of the test sessions. Significance. The ErrP classification results and the possibility of successfully achieving an optimal learning strategy, show the feasibility of the proposed methodology, which allows to move towards clinical experimentation with ASD participants to assess the effectiveness of the approach as hypothesized.

Exploring the Visual Guidance of Motor Imagery in Sustainable Brain–Computer Interfaces

Motor imagery brain–computer interface (MI-BCI) systems hold the possibility of restoring motor function and also offer the possibility of sustainable autonomous living for individuals with various motor and sensory impairments. When utilizing the MI-BCI, the user’s performance impacts the system’s overall accuracy, and concentrating on the user’s mental load enables a better evaluation of the system’s overall performance. The impacts of various levels of abstraction on visual guidance of mental training in motor imagery (MI) may be comprehended. We proposed hypotheses about the effects of visually guided abstraction on brain activity, mental load, and MI-BCI performance, then used the event-related desynchronization (ERD) value to measure the user’s brain activity, extracted the brain power spectral density (PSD) to measure the brain load, and finally classified the left- and right-handed MI through a support vector machine (SVM) classifier. The results showed that visual guidance with a low level of abstraction could help users to achieve the highest brain activity and the lowest mental load, and the highest accuracy rate of MI classification was 97.14%. The findings imply that to improve brain–computer interaction and enable those less capable to regain their mobility, visual guidance with a low level of abstraction should be employed when training brain–computer interface users. We anticipate that the results of this study will have considerable implications for human-computer interaction research in BCI.

Exploring the Ability to Classify Visual Perception and Visual Imagery EEG Data: Toward an Intuitive BCI System

Providing an intuitive interface for the actual use of brain–computer interface (BCI) can increase BCI users’ convenience greatly. We explored the possibility that visual imagery can be used as a paradigm that may constitute a more intuitive, active BCI. To do so, electroencephalography (EEG) data were collected during visual perception and imagery experiments. Three image categories (object, digit, shape) and three different images per category were used as visual stimuli. EEG data from seven subjects were used in this work. Three types of visual perception/imagery EEG data were preprocessed for classification: raw time series data; time–frequency maps; and common spatial pattern (CSP). Five types of classifiers (EEGNet, 1D convolutional neural network (CNN), MultiRocket, MobileNet, support vector machine (SVM)) were applied to each applicable data type among the three preprocessed types. Thus, we investigated the feasibility of classifying three-category or nine-class visual perception/imagery over various classifiers and preprocessed data types. We found that the MultiRocket network showed the best classification performance: yielding approximately 57.02% (max 63.62%) for three-category classification in visual perception and approximately 46.43% (max 71.38%) accuracy for three-category classification in visual imagery. However, no meaningfully improved performance was achieved in the nine-class classification in either visual perception or imagery, although visual perception yielded slightly higher accuracy than visual imagery. From our extensive investigation, we found that visual perception and visual imagery data may be classified; however, it is somewhat doubtful whether either may be applicable to an actual BCI system. It is believed that introducing better-designed advanced deep learning networks together with more informative feature extractions may improve the performance of EEG visual perception/imagery classifications. In addition, a more sophisticated experimental design paradigm may enhance the potential to achieve more intuitive visual imagery BCI.

AtAwAR Translate: Attention-Aware Language Translation Application in Augmented Reality for Mobile Phones.

As lightweight, low-cost EEG headsets emerge, the feasibility of consumer-oriented brain–computer interfaces (BCI) increases. The combination of portable smartphones and easy-to-use EEG dry electrode headbands offers intriguing new applications and methods of human–computer interaction. In previous research, augmented reality (AR) scenarios have been identified to profit from additional user state information—such as that provided by a BCI. In this work, we implemented a system that integrates user attentional state awareness into a smartphone application for an AR written language translator. The attentional state of the user is classified in terms of internally and externally directed attention by using the Muse 2 electroencephalography headband with four frontal electrodes. The classification results are used to adapt the behavior of the translation app, which uses the smartphone’s camera to display translated text as augmented reality elements. We present the first mobile BCI system that uses a smartphone and a low-cost EEG device with few electrodes to provide attention awareness to an AR application. Our case study with 12 participants did not fully support the assumption that the BCI improves usability. However, we are able to show that the classification accuracy and ease of setup are promising paths toward mobile consumer-oriented BCI usage. For future studies, other use cases, applications, and adaptations will be tested for this setup to explore the usability.

Comparing the Usability of Alternative EEG Devices to Traditional Electrode Caps for SSVEP-BCI Controlled Assistive Robots.

Despite having the potential to improve the lives of severely paralyzed users, non-invasive Brain Computer Interfaces (BCI) have yet to be integrated into their daily lives. The widespread adoption of BCI-driven assistive technology is hindered by its lacking usability, as both end-users and researchers alike find fault with traditional EEG caps. In this paper, we compare the usability of four EEG recording devices for Steady-State Visually Evoked Potentials (SSVEP)-BCI applications: an EEG cap (active gel electrodes), two headbands (passive gel or active dry electrodes), and two adhesive electrodes placed on each mastoid. Ten able-bodied participants tested each device by completing an 8-target SSVEP paradigm. Setup times were recorded, and participants rated their satisfaction with each device. The EEG cap obtained the best classification accuracies (Median = 98.96%), followed by the gel electrode headband (Median = 93.75%), and the dry electrode headband (Median = 91.14%). The mastoid electrodes obtained classification accuracies close to chance level (Med = 29.69%). Unknowing of the classification accuracy, participants found the mastoid electrodes to be the most comfortable and discrete. The dry electrode headband obtained the lowest user satisfaction score and was criticized for being too uncomfortable. Participants also noted that the EEG cap was too conspicuous. The gel-based headband provided a good trade-off between BCI performance and user satisfaction.

A Transfer Learning Algorithm to Reduce Brain-Computer Interface Calibration Time for Long-Term Users.

Current motor imagery-based brain-computer interface (BCI) systems require a long calibration time at the beginning of each session before they can be used with adequate levels of classification accuracy. In particular, this issue can be a significant burden for long term BCI users. This article proposes a novel transfer learning algorithm, called r-KLwDSA, to reduce the BCI calibration time for long-term users. The proposed r-KLwDSA algorithm aligns the user's EEG data collected in previous sessions to the few EEG trials collected in the current session, using a novel linear alignment method. Thereafter, the aligned EEG trials from the previous sessions and the few EEG trials from the current sessions are fused through a weighting mechanism before they are used for calibrating the BCI model. To validate the proposed algorithm, a large dataset containing the EEG data from 11 stroke patients, each performing 18 BCI sessions, was used. The proposed framework demonstrated a significant improvement in the classification accuracy, of over 4% compared to the session-specific algorithm, when there were as few as two trials per class available from the current session. The proposed algorithm was particularly successful in improving the BCI accuracy of the sessions that had initial session-specific accuracy below 60%, with an average improvement of around 10% in the accuracy, leading to more stroke patients having meaningful BCI rehabilitation.

Studies to Overcome Brain–Computer Interface Challenges

A brain–computer interface (BCI) is a promising technology that can analyze brain signals and control a robot or computer according to a user’s intention. This paper introduces our studies to overcome the challenges of using BCIs in daily life. There are several methods to implement BCIs, such as sensorimotor rhythms (SMR), P300, and steady-state visually evoked potential (SSVEP). These methods have different pros and cons according to the BCI type. However, all these methods are limited in choice. Controlling the robot arm according to the intention enables BCI users can do various things. We introduced the study predicting three-dimensional arm movement using a non-invasive method. Moreover, the study was described compensating the prediction using an external camera for high accuracy. For daily use, BCI users should be able to turn on or off the BCI system because of the prediction error. The users should also be able to change the BCI mode to the efficient BCI type. The BCI mode can be transformed based on the user state. Our study was explained estimating a user state based on a brain’s functional connectivity and a convolutional neural network (CNN). Additionally, BCI users should be able to do various tasks, such as carrying an object, walking, or talking simultaneously. A multi-function BCI study was described to predict multiple intentions simultaneously through a single classification model. Finally, we suggest our view for the future direction of BCI study. Although there are still many limitations when using BCI in daily life, we hope that our studies will be a foundation for developing a practical BCI system.

Two sides of the same coin: adaptation of BCIs to internal states with user-centered design and electrophysiological features

ABSTRACT The ideal brain–computer interface (BCI) adapts to the user’s state to enable optimal BCI performance. Two methods of BCI adaptation are commonly applied: User-centered design (UCD) responds to individual user needs and requirements. Passive BCIs can adapt via online analysis of electrophysiological signals. Despite similar goals, these methods are rarely discussed in combination. Hence, we organized a workshop for the 8th International BCI Meeting 2021 to discuss the combined application of both methods. Here we expand upon the workshop by discussing UCD in more detail regarding its utility for end-users as well as non-end-user-based early-stage BCI development. Furthermore, we explore electrophysiology-based online user state adaptation concerning consciousness and pain detection. The integration of the numerous BCI user state adaptation methods into a unified process remains challenging. Yet, further systematic accumulation of specific knowledge about assessment and integration of internal user states bears great potential for BCI optimization.

Workshops of the eighth international brain–computer interface meeting: BCIs: the next frontier

ABSTRACT The Eighth International Brain–Computer Interface (BCI) Meeting was held June 7–9, 2021 in a virtual format. The conference continued the BCI Meeting series’ interactive nature with 21 workshops covering the breadth of topics in BCI (also called brain–machine interface) research. Some workshops provided detailed examinations of methods, hardware, or processes. Others focused on BCI applications or user groups. Several workshops continued consensus building efforts designed to create BCI standards and improve comparisons between studies and the potential for meta-analysis and large multi-site clinical trials. Ethical and translational considerations were the primary topic for some workshops or an important secondary consideration for others. The range of BCI applications continues to expand, with more workshops focusing on approaches that can extend beyond the needs of those with physical impairments. This paper summarizes each workshop, provides background information and references for further study, summarizes discussions, and describes the resulting conclusion, challenges, or initiatives.

Reducing the Calibration Time in Somatosensory BCI by Using Tactile ERD.

We propose a tactile-induced-oscillation approach to reduce the calibration time in somatosensory brain-computer interfaces (BCI). Based on the similarity between tactile induced event-related desynchronization (ERD) and imagined sensation induced ERD activation, we extensively evaluated BCI performance when using a conventional and a novel calibration strategy. In the conventional calibration, the tactile imagined data was used, while in the sensory calibration model sensory stimulation data was used. Subjects were required to sense the tactile stimulus when real tactile was applied to the left or right wrist and were required to perform imagined sensation tasks in the somatosensory BCI paradigm. The sensory calibration led to a significantly better performance than the conventional calibration when tested on the same imagined sensation dataset ( [Formula: see text]=10.89, P=0.0038), with an average 5.1% improvement in accuracy. Moreover, the sensory calibration was 39.3% faster in reaching a performance level of above 70% accuracy. The proposed approach of using tactile ERD from the sensory cortex provides an effective way of reducing the calibration time in a somatosensory BCI system. The tactile stimulation would be specifically useful before BCI usage, avoiding excessive fatigue when the mental task is difficult to perform. The tactile ERD approach may find BCI applications for patients or users with preserved afferent pathways.

Performance Variation of a Somatosensory BCI Based on Imagined Sensation: A Large Population Study.

A proportion of users cannot achieve adequate brain-computer interface (BCI) control. The diversity of BCI modalities provides a way to solve this emerging issue. Here, we investigate the accuracy of a somatosensory BCI based on sensory imagery (SI). During the SI tasks, subjects were instructed to imagine a tactile sensation and to maintain the attention on the corresponding hand, as if there was tactile stimulus on the skin of the wrist. The performance across 106 healthy subjects in left- and right-hand SI discrimination was 78.9±13.2%. In 70.7% of the subjects the performance was above 70%. The SI task induced a contralateral cortical activation, and high-density EEG source localization showed that the real tactile stimulation and imagined tactile stimulation shared similar cortical activations within the somatosensory cortex. The somatosensory BCI based on SI provides a new signal modality for independent BCI development. Moreover, a combination of SI and other BCI modalities, such as motor imagery, may provide new avenues for further improving BCI usage and applicability, especially in those subjects unable to attain adequate BCI control with conventional BCI modalities.

Thoughts Unlocked by Technology\u2014a Survey in Germany About Brain-Computer Interfaces

A brain-computer interface (BCI) is a rapidly evolving neurotechnology connecting the human brain with a computer. In its classic form, brain activity is recorded and used to control external devices like protheses or wheelchairs. Thus, BCI users act with the power of their thoughts. While the initial development has focused on medical uses of BCIs, non-medical applications have recently been gaining more attention, for example in automobiles, airplanes, and the entertainment context. However, the attitudes of the general public towards BCIs have hardly been explored. Among the general population in Germany aged 18–65 years, a representative online survey with 20 items was conducted in summer 2018 (n = 1000) and analysed by descriptive statistics. The survey assessed: affinity for technology; previous knowledge and experience concerning BCIs; the attitude towards ethical, social and legal implications of BCI use and demographic information. Our results indicate that BCIs are a unique and puzzling way of human–machine interaction. The findings reveal a positive view and high level of trust in BCIs on the one hand but on the other hand a wide range of ethical and anthropological concerns. Agency and responsibility were clearly attributed to the BCI user. The participants’ opinions were divided regarding the impact BCIs have on humankind. In summary, a high level of ambivalence regarding BCIs was found. We suggest better information of the public and the promotion of public deliberation about BCIs in order to ensure responsible development and application of this potentially disruptive technology.

The dorsolateral pre-frontal cortex bi-polar error-related potential in a locked-in patient implanted with a daily use brain\u2013computer interface

While brain computer interfaces (BCIs) offer the potential of allowing those suffering from loss of muscle control to once again fully engage with their environment by bypassing the affected motor system and decoding user intentions directly from brain activity, they are prone to errors. One possible avenue for BCI performance improvement is to detect when the BCI user perceives the BCI to have made an unintended action and thus take corrective actions. Error-related potentials (ErrPs) are neural correlates of error awareness and as such can provide an indication of when a BCI system is not performing according to the user’s intentions. Here, we investigate the brain signals of an implanted BCI user suffering from locked-in syndrome (LIS) due to late-stage ALS that prevents her from being able to speak or move but not from using her BCI at home on a daily basis to communicate, for the presence of error-related signals. We first establish the presence of an ErrP originating from the dorsolateral pre-frontal cortex (dLPFC) in response to errors made during a discrete feedback task that mimics the click-based spelling software she uses to communicate. Then, we show that this ErrP can also be elicited by cursor movement errors in a continuous BCI cursor control task. This work represents a first step toward detecting ErrPs during the daily home use of a communications BCI.

Vigilance Estimating in SSVEP-Based BCI Using Multimodal Signals.

Brain-computer interface (BCI) is a communication system that allows a direct connection between the human brain and external devices. With the application of BCI, it is important to estimate vigilance for BCI users. In order to investigate the vigilance changes of the subjects during BCI tasks and develop a multimodal method to estimate the vigilance level, a high-speed 4-target BCI system for cursor control was built based on steady-state visual evoked potential (SSVEP). 18 participants were recruited and underwent a 90-min continuous cursor-control BCI task, when electroencephalogram (EEG), electrooculogram (EOG), electrocardiography (ECG), and electrodermal activity (EDA) were recorded simultaneously. Then, we extracted features from the multimodal signals and applied regression models to estimate vigilance. Experimental results showed that the differential entropy (DE) feature could effectively reflect the change of vigilance. The vigilance estimation method, which integrates DE and EOG features into the support vector regression (SVR) model, achieved a better performance than the compared methods. These results demonstrate the feasibility of our methods for estimating vigilance levels in BCI.

The impact of distractions on intracortical brain–computer interface control of a robotic arm

ABSTRACT The objective of the study was to evaluate the impact of distractions on intracortical brain–computer interface (BCI) performance. Two individuals with tetraplegia had microelectrode arrays implanted into their motor cortex. Each used a BCI to move a robotic arm between two targets as quickly as possible, performed alone and with distractions. Performance outcomes were similar across conditions, except for a drop in the median path efficiency in the motor action distraction for Subject P2 (p = 0.033, Mann–Whitney U-test). Other distraction conditions resulted in intermittent impairments in performance, but were not statistically different from the no distraction condition. Both subjects rated the overall difficulty of the task with and without distractions as low. Consistent levels of BCI performance were mirrored by small differences in neural firing rates across conditions, demonstrating that experienced BCI users can maintain performance on a simple reaching task under these various conditions.

Single-Channel Selection for Detecting Steady-State Visual Evoked Potentials in a Brain-Computer Interface Speller

Purpose: Brain-Computer Interface (BCI) provides a secondary communication pathway for patients with neuromuscular diseases such as amyotrophic lateral sclerosis (ALS) or brainstem stroke in which they are almost incapacitated to move or talk. BCI enacts neural oscillations to generate a command signal for machines to operate desired tasks instead of patients. Steady-State Visual Evoked Potential (SSVEP) is the brain response to a visual stimulus, with the same frequency as its eliciting signal (or its harmonics), that has been widely used in BCI environments. In order to provide a more convenient situation for BCI users, we aim to find the best single-channel EEG, which results in the highest accuracy for detecting SSVEP.
 Materials and Methods: We developed a Deep Convolutional Neural Network with single-channel EEG as input to classify a 40-class SSVEP; each class represents a stimulus, which has been acquired from 35 subjects. We used 3.5 s windows of the data (Trials of 3.5 seconds length for each class) to train our model and leave-one-subject-out cross-validation for the testing.
 Results: The proposed method resulted in the average classification accuracy of 74.30%±20.85 and Information Transfer Rate (ITR) of 57.51 bpm which outperforms the previous single-channel SSVEP BCIs in terms of ITR. Also, the O1 channel achieved the best performance criteria among the channels in the occipital and parietal lobes, which seems reasonable according to previous researches for finding the location of neurons, responsible for visual tasks in the brain.
 Conclusion: In this study, we dedicated our efforts to reduce the number of EEG channels to a single channel while proposing a deep learning strategy for an SSVEP-based BCI speller to make it more feasible for patients whose lives are dependent on such systems. The overall results, although not ideal, open a new promising window toward a feasible BCI system.

Improving motor imagery classification during induced motor perturbations

Objective. Motor imagery is the mental simulation of movements. It is a common paradigm to design brain-computer interfaces (BCIs) that elicits the modulation of brain oscillatory activity similar to real, passive and induced movements. In this study, we used peripheral stimulation to provoke movements of one limb during the performance of motor imagery tasks. Unlike other works, in which induced movements are used to support the BCI operation, our goal was to test and improve the robustness of motor imagery based BCI systems to perturbations caused by artificially generated movements. Approach. We performed a BCI session with ten participants who carried out motor imagery of three limbs. In some of the trials, one of the arms was moved by neuromuscular stimulation. We analysed 2-class motor imagery classifications with and without movement perturbations. We investigated the performance decrease produced by these disturbances and designed different computational strategies to attenuate the observed classification accuracy drop. Main results. When the movement was induced in a limb not coincident with the motor imagery classes, extracting oscillatory sources of the movement imagination tasks resulted in BCI performance being similar to the control (undisturbed) condition; when the movement was induced in a limb also involved in the motor imagery tasks, the performance drop was significantly alleviated by spatially filtering out the neural noise caused by the stimulation. We also show that the loss of BCI accuracy was accompanied by weaker power of the sensorimotor rhythm. Importantly, this residual power could be used to predict whether a BCI user will perform with sufficient accuracy under the movement disturbances. Significance. We provide methods to ameliorate and even eliminate motor related afferent disturbances during the performance of motor imagery tasks. This can help improving the reliability of current motor imagery based BCI systems.

The BCI Glossary: a first proposal for a community review

ABSTRACT The description of Brain-Computer Interfaces (BCI) can lead to confusion because of the high heterogeneity of devices, protocols, and applications. Besides, different professional categories are involved: end-users, clinicians, therapists, and engineers; each one having different conceptions of BCI-related terms. This can cause misunderstandings and errors, and it makes it impossible to compare different systems and their performances. The IEEE P2731 working group has been working on a standardized glossary for BCI research, together with a functional model for BCI. Here, we are presenting a first version of the BCI glossary, generated by the collective effort of the working group. One hundred fifty-three terms have been identified to be critical for describing in a standardized way BCI systems and their related aspects (e.g., the neurophysiological characteristics of the neural signals recorded). Each term has been provided with a definition, merged from multiple ones proposed by working group members, with appropriate references to the current state of the art. Finally, we are asking for feedback and suggestions about this first version of the BCI glossary to the wider community of BCI users and researchers. External inputs will improve the glossary, which will become, after further revisions, an official IEEE standard.

User State Classification Based on Functional Brain Connectivity Using a Convolutional Neural Network

The brain–computer interface (BCI) is a promising technology where a user controls a robot or computer by thinking with no movement. There are several underlying principles to implement BCI, such as sensorimotor rhythms, P300, steady-state visually evoked potentials, and directional tuning. Generally, different principles are applied to BCI depending on the application, because strengths and weaknesses vary according to each BCI method. Therefore, BCI should be able to predict a user state to apply suitable principles to the system. This study measured electroencephalography signals in four states (resting, speech imagery, leg-motor imagery, and hand-motor imagery) from 10 healthy subjects. Mutual information from 64 channels was calculated as brain connectivity. We used a convolutional neural network to predict a user state, where brain connectivity was the network input. We applied five-fold cross-validation to evaluate the proposed method. Mean accuracy for user state classification was 88.25 ± 2.34%. This implies that the system can change the BCI principle using brain connectivity. Thus, a BCI user can control various applications according to their intentions.