Active Brain-computer Interfaces Research Articles

POMDP-BCI: A Benchmark of (Re)Active BCI Using POMDP to Issue Commands.

Past research in Brain-Computer Interfaces (BCI) have presented different decoding algorithms for different modalities. Meanwhile, highly specific decision making processes have been developed for some of these modalities, while others lack such a component in their classic pipeline. The present work proposes a model based on Partially Observable Markov Decission Process (POMDP) that works as a high-level decision making framework for three different active/reactive BCI modalities. We tested our approach on three different BCI modalities using publicly available datasets. We compared the general POMDP model as a decision making process with state of the art methods for each BCI modality. Accuracy, false positive (FP) trials, no-action (NA) trials and average decision time are presented as metrics. Our results show how the presented POMDP models achieve comparable or better performance to the presented baseline methods, while being usable for the three proposed experiments without significant changes. Crucially, it offers the possibility of taking no-action (NA) when the decoding does not perform well. The present work implements a flexible POMDP model that acts as a sequential decision framework for BCI systems that lack such a component, and perform comparably to those that include it. We believe the proposed POMDP framework provides several interesting properties for future BCI developments, mainly the generalizability to any BCI modality and the possible integration of other physiological or brain data pipelines under a unified decision-making framework.

Wearable brain-computer interface technology and its application

Wearable Brain-Computer Interfaces (BCIs) signify a critical evolution in human-machine communication, driven by the convergence of neuroscience, engineering, and information technology. With applications that span across industrial, medical, and recreational domains, BCIs hold potential to redefine our interaction with the technological landscape. This manuscript elucidates this transformative juncture, bifurcating into passive and active BCIs. In passive BCIs, innovations leveraging Virtual Reality (VR) and Augmented Reality (AR) are delineated, progress has been demonstrated in the classification of passively induced emotional signals, highlighting the emergence of hands-free control systems like quadcopter control and industrial inspections. The discourse on active BCIs unveils advancements such as real-time handwriting and speech decoding, driver drowsiness detection, and emotion recognition, aided by machine learning techniques. Despite groundbreaking progress, challenges in algorithm optimization, adaptability, multimodal signal complexity, and ethics persist. Future directions emphasize the potential of deep learning and multimode input signals collaboration. The manuscript underscores the societal implications, particularly in rehabilitation, communication, and entertainment. The review, therefore, serves as both an appraisal and a roadmap for the burgeoning field of wearable BCIs, underlining its role as a pathway to enhance human capabilities and quality of life.

Long-Range α-Synchronization as Control Signal for BCI: A Feasibility Study.

Shifts in spatial attention are associated with variations in α band (α, 8-14 Hz) activity, specifically in interhemispheric imbalance. The underlying mechanism is attributed to local α-synchronization, which regulates local inhibition of neural excitability, and frontoparietal synchronization reflecting long-range communication. The direction-specific nature of this neural correlate brings forward its potential as a control signal in brain-computer interfaces (BCIs). In the present study, we explored whether long-range α-synchronization presents lateralized patterns dependent on voluntary attention orienting and whether these neural patterns can be picked up at a single-trial level to provide a control signal for active BCI. We collected electroencephalography (EEG) data from a cohort of healthy adults (n = 10) while performing a covert visuospatial attention (CVSA) task. The data show a lateralized pattern of α-band phase coupling between frontal and parieto-occipital regions after target presentation, replicating previous findings. This pattern, however, was not evident during the cue-to-target orienting interval, the ideal time window for BCI. Furthermore, decoding the direction of attention trial-by-trial from cue-locked synchronization with support vector machines (SVMs) was at chance level. The present findings suggest EEG may not be capable of detecting long-range α-synchronization in attentional orienting on a single-trial basis and, thus, highlight the limitations of this metric as a reliable signal for BCI control.

A survey on encoding and decoding technology of non-invasive brain-computer interface

A survey on encoding and decoding technology of non-invasive brain-computer interface

Hybrid mental tasks based human computer interface via integration of pronunciation and motor imagery

Objective. Among the existing active brain–computer interfaces (BCI), the motor imagination (MI) is widely used. To operate the MI BCI effectively, subjects need to carry out trainings on corresponding imagining tasks. Here, we studied how to reduce the discomfort and fatigue of active BCI imaginary tasks and the inability to concentrate on them while improving the accuracy. Approach. This paper proposes a hybrid BCI composed of MI and pronunciation imagination (PI). The electroencephalogram signals of ten subjects are recognized by the adaptive Riemannian distance classification and the improved frequency selective filter-bank Common Spatial Pattern recognition. Main results. The results show that under the new paradigm with the combination of MI and PI, the recognition accuracy is higher than the MI alone. The highest recognition rate of the proposed hybrid system can reach more than 90%. Furthermore, through the subjects’ scoring results of the operation difficulty, it is concluded that the designed hybrid paradigm is more operable than the traditional BCI paradigm. Significance. The separable tasks in the active BCI are limited and the accuracy needs to be improved. The new hybrid paradigm proposed by us improves the accuracy and operability of the active BCI system, providing a new possibility for the research direction of the active BCI.

Exploring the Ability to Classify Visual Perception and Visual Imagery EEG Data: Toward an Intuitive BCI System

Providing an intuitive interface for the actual use of brain–computer interface (BCI) can increase BCI users’ convenience greatly. We explored the possibility that visual imagery can be used as a paradigm that may constitute a more intuitive, active BCI. To do so, electroencephalography (EEG) data were collected during visual perception and imagery experiments. Three image categories (object, digit, shape) and three different images per category were used as visual stimuli. EEG data from seven subjects were used in this work. Three types of visual perception/imagery EEG data were preprocessed for classification: raw time series data; time–frequency maps; and common spatial pattern (CSP). Five types of classifiers (EEGNet, 1D convolutional neural network (CNN), MultiRocket, MobileNet, support vector machine (SVM)) were applied to each applicable data type among the three preprocessed types. Thus, we investigated the feasibility of classifying three-category or nine-class visual perception/imagery over various classifiers and preprocessed data types. We found that the MultiRocket network showed the best classification performance: yielding approximately 57.02% (max 63.62%) for three-category classification in visual perception and approximately 46.43% (max 71.38%) accuracy for three-category classification in visual imagery. However, no meaningfully improved performance was achieved in the nine-class classification in either visual perception or imagery, although visual perception yielded slightly higher accuracy than visual imagery. From our extensive investigation, we found that visual perception and visual imagery data may be classified; however, it is somewhat doubtful whether either may be applicable to an actual BCI system. It is believed that introducing better-designed advanced deep learning networks together with more informative feature extractions may improve the performance of EEG visual perception/imagery classifications. In addition, a more sophisticated experimental design paradigm may enhance the potential to achieve more intuitive visual imagery BCI.

Signal analysis and classification of a novel active brain-computer interface based on four-category sequential coding

In order to increase the number of instruction sets for the active brain-computer interface (BCI), a novel experimental paradigm based on the sequential coding of motor imagery and speech imagery is proposed in this paper. By dividing one motor imagery and one speech imagery into time series, four imagination tasks are obtained: 1) motor imagery; 2) speech imagery; 3) motor imagery first and then speech imagery; 4) speech imagery first and then motor imagery. After analyzing the temporal, frequency and spatial features of electroencephalography (EEG) signals, four types of signals are classified by a multi-classification model based on time sequence. In this model, feature extraction and classification are accomplished by common spatial pattern (CSP) and support vector machine (SVM), respectively. 12 subjects participate in this experimental paradigm, and their average classification accuracy is 68.94%. The classification results are much higher than random probability, so the proposed experimental paradigm is feasible and valuable. The experimental paradigm based on sequential coding can effectively increase the number of instruction sets of active BCIs, so the practicability of BCIs is also improved.

Deep Learning Model With Adaptive Regularization for EEG-Based Emotion Recognition Using Temporal and Frequency Features

Since EEG signal acquisition is non-invasive and portable, it is convenient to be used for different applications. Recognizing emotions based on Brain-Computer Interface (BCI) is an important active BCI paradigm for recognizing the inner state of persons. There are extensive studies about emotion recognition, most of which heavily rely on staged complex handcrafted EEG feature extraction and classifier design. In this paper, we propose a hybrid multi-input deep model with convolution neural networks (CNNs) and bidirectional Long Short-term Memory (Bi-LSTM). CNNs extract time-invariant features from raw EEG data, and Bi-LSTM allows long-range lateral interactions between features. First, we propose a novel hybrid multi-input deep learning approach for emotion recognition from raw EEG signals. Second, in the first layers, we use two CNNs with small and large filter sizes to extract temporal and frequency features from each raw EEG epoch of 62-channel 2-s and merge with differential entropy of EEG band. Third, we apply the adaptive regularization method over each parallel CNN’s layer to consider the spatial information of EEG acquisition electrodes. The proposed method is evaluated on two public datasets, SEED and DEAP. Our results show that our technique can significantly improve the accuracy in comparison with the baseline where no adaptive regularization techniques are used.

Multi-Parameter Physiological State Monitoring in Target Detection Under Real-World Settings.

Mental state changes induced by stimuli under experimental settings or by daily events in real life affect task performance and are entwined with physical and mental health. In this study, we developed a physiological state indicator with five parameters that reflect the subject’s real-time physiological states based on online EEG signal processing. These five parameters are attention, fatigue, stress, and the brain activity shifts of the left and right hemispheres. We designed a target detection experiment modified by a cognitive attention network test for validating the effectiveness of the proposed indicator, as such conditions would better approximate a real chaotic environment. Results demonstrated that attention levels while performing the target detection task were significantly higher than during rest periods, but also exhibited a decay over time. In contrast, the fatigue level increased gradually and plateaued by the third rest period. Similar to attention levels, the stress level decreased as the experiment proceeded. These parameters are therefore shown to be highly correlated to different stages of the experiment, suggesting their usage as primary factors in passive brain-computer interfaces (BCI). In addition, the left and right brain activity indexes reveal the EEG neural modulations of the corresponding hemispheres, which set a feasible reference of activation for an active BCI control system, such as one executing motor imagery tasks. The proposed indicator is applicable to potential passive and active BCI applications for monitoring the subject’s physiological state change in real-time, along with providing a means of evaluating the associated signal quality to enhance the BCI performance.

Multi-Scale Neural Network for EEG Representation Learning in BCI

Recent advances in deep learning have had a methodological and practical impact on brain-computer interface (BCI) research. Among the various deep network architectures, convolutional neural networks (CNNs) have been well suited for spatio-spectral-temporal electroencephalogram (EEG) signal representation learning. Most of the existing CNN-based methods described in the literature extract features at a sequential level of abstraction with repetitive nonlinear operations and involve densely connected layers for classification. However, studies in neurophysiology have revealed that EEG signals carry information in different ranges of frequency components. To better reflect these multi-frequency properties in EEGs, we propose a novel deep multi-scale neural network that discovers feature representations in multiple frequency/time ranges and extracts relationships among electrodes, i.e., spatial representations, for subject intention/condition identification. Furthermore, by completely representing EEG signals with spatio-spectral-temporal information, the proposed method can be utilized for diverse paradigms in both active and passive BCIs, contrary to existing methods that are primarily focused on single-paradigm BCIs. To demonstrate the validity of our proposed method, we conducted experiments on various paradigms of active/passive BCI datasets. Our experimental results demonstrated that the proposed method achieved performance improvements when judged against comparable state-of-the-art methods. Additionally, we analyzed the proposed method using different techniques, such as PSD curves and relevance score inspection to validate the multi-scale EEG signal information capturing ability, activation pattern maps for investigating the learned spatial filters, and t-SNE plotting for visualizing represented features. Finally, we also demonstrated our method's application to real-world problems. Based on our experimental results and analyses, we believe that the proposed multi-scale neural network can be useful for various BCI paradigms, as a starting model or as a backbone network in any new BCI experiments.

Prefrontal Asymmetry BCI Neurofeedback Datasets.

Prefrontal cortex (PFC) asymmetry is an important marker in affective neuroscience and has attracted significant interest, having been associated with studies of motivation, eating behavior, empathy, risk propensity, and clinical depression. The data presented in this paper are the result of three different experiments using PFC asymmetry neurofeedback (NF) as a Brain-Computer Interface (BCI) paradigm, rather than a therapeutic mechanism aiming at long-term effects, using functional near-infrared spectroscopy (fNIRS) which is known to be particularly well-suited to the study of PFC asymmetry and is less sensitive to artifacts. From an experimental perspective the BCI context brings more emphasis on individual subjects' baselines, successful and sustained activation during epochs, and minimal training. The subject pool is also drawn from the general population, with less bias toward specific behavioral patterns, and no inclusion of any patient data. We accompany our datasets with a detailed description of data formats, experiment and protocol designs, as well as analysis of the individualized metrics for definitions of success scores based on baseline thresholds as well as reference tasks. The work presented in this paper is the result of several experiments in the domain of BCI where participants are interacting with continuous visual feedback following a real-time NF paradigm, arising from our long-standing research in the field of affective computing. We offer the community access to our fNIRS datasets from these experiments. We specifically provide data drawn from our empirical studies in the field of affective interactions with computer-generated narratives as well as interfacing with algorithms, such as heuristic search, which all provide a mechanism to improve the ability of the participants to engage in active BCI due to their realistic visual feedback. Beyond providing details of the methodologies used where participants received real-time NF of left-asymmetric increase in activation in their dorsolateral prefrontal cortex (DLPFC), we re-establish the need for carefully designing protocols to ensure the benefits of NF paradigm in BCI are enhanced by the ability of the real-time visual feedback to adapt to the individual responses of the participants. Individualized feedback is paramount to the success of NF in BCIs.

Separable EEG Features Induced by Timing Prediction for Active Brain-Computer Interfaces.

Brain–computer interfaces (BCI) have witnessed a rapid development in recent years. However, the active BCI paradigm is still underdeveloped with a lack of variety. It is imperative to adapt more voluntary mental activities for the active BCI control, which can induce separable electroencephalography (EEG) features. This study aims to demonstrate the brain function of timing prediction, i.e., the expectation of upcoming time intervals, is accessible for BCIs. Eighteen subjects were selected for this study. They were trained to have a precise idea of two sub-second time intervals, i.e., 400 ms and 600 ms, and were asked to measure a time interval of either 400 ms or 600 ms in mind after a cue onset. The EEG features induced by timing prediction were analyzed and classified using the combined discriminative canonical pattern matching and common spatial pattern. It was found that the ERPs in low-frequency (0~4 Hz) and energy in high-frequency (20~60 Hz) were separable for distinct timing predictions. The accuracy reached the highest of 93.75% with an average of 76.45% for the classification of 400 vs. 600 ms timing. This study first demonstrates that the cognitive EEG features induced by timing prediction are detectable and separable, which is feasible to be used in active BCIs controls and can broaden the category of BCIs.

Online detection of error-related potentials in multi-class cognitive task-based BCIs

ABSTRACTOne method for improving the accuracy and hence the rate of communication of a brain–computer interface (BCI) is to automatically correct erroneous classifications by exploiting error-related potentials (ErrPs). The merit of such a correction scheme has been demonstrated in both active (e.g. motor imagery) and reactive (e.g. P300) BCIs. Here, we investigated the effect of ErrP-guided error correction in a three-class, active BCI based on cognitive rather than motor imagery tasks using electroencephalography (EEG). Ten able-bodied adults participated in three sessions of data collection. For each participant, a ternary BCI differentiated among idle state and two personally selected cognitive tasks (e.g. mental arithmetic, counting, word generation, and figure rotation). Real-time feedback of the BCI decision was displayed to the participant following each task. EEG data after feedback onset were used to detect ErrPs and correct the BCI’s output in the case of detected errors. ErrP-based error correction modestly but significantly improved the average online task classification accuracy (+7%) as well as the information transfer rate (+0.9 bits/min) of the ternary BCI across participants. Our findings support further study of ErrPs in active BCIs based on cognitive tasks.

Exploiting error-related potentials in cognitive task based BCI

Brain-computer interfaces (BCIs) can make mistakes in recognizing user intention. It has been well-established that for reactive and motor imagery (MI) BCIs, an error-related potential (ErrP) occurs subsequent to such mistakes, and can be used to improve BCI performance. However, the presence of ErrPs in active BCIs based on non-MI cognitive tasks has not been confirmed. In this study, we attempted to elicit ErrPs in a BCI based on non-MI mental tasks. Twelve typically developed young adults participated in two sessions each. Participants performed multiple iterations of five different mental tasks (mental arithmetic, counting, word generation, figure rotation and idle state) to ‘knock down’ one of 4 targets on a graphical interface (each mental task was associated with a different target). To simulate errors, a random subset of 20% of the trials were followed by incorrect feedback (i.e., the wrong target fell). Our findings confirmed the presence of an interaction ErrP, with a negative peak at ∼180 ms, followed by two positive peaks, respectively, at ∼400 and ∼630 ms post-feedback onset. The classification of mental tasks and error versus non-error trials were both performed using a pseudo-online paradigm where the last quarter of trials were used for testing. For binary (task and idle) and ternary (2 tasks and idle) classification, across-participant average accuracies of 76% ± 12 and 63%±12, respectively, were attained. An average area under curve (AUC) of 0.83 was reached across participants for the detection of ErrPs. After applying ErrP-based error correction, the average binary and ternary classification accuracies of mental tasks improved by 9% and 14%, respectively. Our findings support the addition of ErrP detection and ErrP-informed correction to maximize the accuracy of BCIs based on cognitive tasks.

Idle state classification using spiking activity and local field potentials in a brain computer interface.

Previous studies of intracortical brain-computer interfaces (BCIs) have often focused on or compared the use of spiking activity and local field potentials (LFPs) for decoding kinematic movement parameters. Conversely, using these signals to detect the initial intention to use a neuroprosthetic device or not has remained a relatively understudied problem. In this study, we examined the relative performance of spiking activity and LFP signals in detecting discrete state changes in attention regarding a user's desire to actively control a BCI device. Preliminary offline results suggest that the beta and high gamma frequency bands of LFP activity demonstrated a capacity for discriminating idle/active BCI control states equal to or greater than firing rate activity on the same channel. Population classifier models using either signal modality demonstrated an indistinguishably high degree of accuracy in decoding rest periods from active BCI reach periods as well as other portions of active BCI task trials. These results suggest that either signal modality may be used to reliably detect discrete state changes on a fine time scale for the purpose of gating neural prosthetic movements.

Investigation of engagement of viewers in movie trailers using electroencephalography

Brain-computer interfaces (BCIs) have been focused on providing direct communications to the disabled. Recently, BCI researchers have expanded BCI applications to non-medical uses and categorized them as active BCI, reactive BCI, and passive BCI. Neurocinematics, a new application of reactive BCIs, aims to understand viewers’ cognitive and affective responses to movies from neural activity, providing more objective information than traditional subjective self-reports. However, studies on analytical indices for neurocinematics have verified their indices by comparisons with self-reports. To overcome this contradictory issue, we proposed using an independent psychophysical index to evaluate a neural engagement index (NEI). We made use of the secondary task reaction time (STRT), which measures participants’ engagement in a primary task by their reaction time to a secondary task; here, responding to a tactile stimulus was the secondary task and watching a movie trailer was the primary task. NEI was developed as changes in the difference between frontal beta and alpha activity of EEG. We evaluated movie trailers using NEI, STRT, and self-reports and found a significant correlation between STRT and NEI across trailers but no correlation between any of the self-report results and STRT or NEI. Our results suggest that NEI developed for neurocinematics may conform well with more objective psychophysical assessments but not with subjective self-reports.

Differentiating closed-loop cortical intention from rest: building an asynchronous electrocorticographic BCI

Objective. Recent experiments have shown that electrocorticography (ECoG) can provide robust control signals for a brain–computer interface (BCI). Strategies that attempt to adapt a BCI control algorithm by learning from past trials often assume that the subject is attending to each training trial. Likewise, automatic disabling of movement control would be desirable during resting periods when random brain fluctuations might cause unintended movements of a device. To this end, our goal was to identify ECoG differences that arise between periods of active BCI use and rest. Approach. We examined spectral differences in multi-channel, epidural micro-ECoG signals recorded from non-human primates when rest periods were interleaved between blocks of an active BCI control task. Main Results. Post-hoc analyses demonstrated that these states can be decoded accurately on both a trial-by-trial and real-time basis, and this discriminability remains robust over a period of weeks. In addition, high gamma frequencies showed greater modulation with desired movement direction, while lower frequency components demonstrated greater amplitude differences between task and rest periods, suggesting possible specialized BCI roles for these frequencies. Significance. The results presented here provide valuable insight into the neurophysiology of BCI control as well as important considerations toward the design of an asynchronous BCI system.

Cortical Adaptation to a Chronic Micro-Electrocorticographic Brain Computer Interface

Brain-computer interface (BCI) technology decodes neural signals in real time to control external devices. In this study, chronic epidural micro-electrocorticographic recordings were performed over primary motor (M1) and dorsal premotor (PMd) cortex of three macaque monkeys. The differential gamma-band amplitude (75-105 Hz) from two arbitrarily chosen 300 μm electrodes (one located over each cortical area) was used for closed-loop control of a one-dimensional BCI device. Each monkey rapidly learned over a period of days to successfully control the velocity of a computer cursor. While both cortical areas contributed to success on the BCI task, the control signals from M1 were consistently modulated more strongly than those from PMd. Additionally, we observe that gamma-band power during active BCI control is always above resting brain activity. This suggests that purposeful gamma-band modulation is an active process that is obtained through increased cortical activation.

Toward Brain-Actuated Humanoid Robots: Asynchronous Direct Control Using an EEG-Based BCI

The brain-computer interface (BCI) technique is a novel control interface to translate human intentions into appropriate motion commands for robotic systems. The aim of this study is to apply an asynchronous direct-control system for humanoid robot navigation using an electroencephalograph (EEG), based active BCI. The experimental procedures consist of offline training, online feedback testing, and real-time control sessions. The amplitude features from EEGs are extracted using power spectral analysis, while informative feature components are selected based on the Fisher ratio. The two classifiers are hierarchically structured to identify human intentions and trained to build an asynchronous BCI system. For the performance test, five healthy subjects controlled a humanoid robot navigation to reach a target goal in an indoor maze by using their EEGs based on real-time images obtained from a camera on the head of the robot. The experimental results showed that the subjects successfully controlled the humanoid robot in the indoor maze and reached the goal by using the proposed asynchronous EEG-based active BCI system.