Future Brain-computer Interface Research Articles

Direction decoding of imagined hand movements using subject-specific features from parietal EEG

Objective. Research on the decoding of brain signals to control external devices is rapidly emerging due to its versatile potential applications, including neuroprosthetic control and neurorehabilitation. Electroencephalogram (EEG)-based non-invasive brain–computer interface (BCI) systems decode brain signals to establish an augmented communication and control pathway between the brain and the computer. The development of an efficient BCI system requires accurate decoding of neural activity underlying the user’s intentions. This study investigates the directional tuning of EEG characteristics from the posterior parietal region, associated with bidirectional hand movement imagination or motor imagery (MI) in left and right directions. Approach. The imagined movement directions of the chosen hand were decoded using a combination of envelope and phase features derived from parietal EEGs of both hemispheres. The proposed algorithm uses wavelets for spectral decomposition, and discriminative subject-specific subband levels are identified based on Fisher analysis of envelope and phase features. The selected features from the discriminative subband levels are used to classify left and right MI directions of the hand using a support vector machine classifier. Furthermore, the performance of the proposed algorithm is evaluated by incorporating a maximum-variance-based EEG time bin selection algorithm. Main results. With the time bin selection approach using subject-specific features, the proposed algorithm yielded an average left vs right MI direction decoding accuracy of 73.33% across 15 healthy subjects. In addition, the decoding accuracy offered by the phase features was higher than that of the envelope features, indicating the importance of phase features in MI kinematics decoding. Significance. The results reveal the significance of the parietal EEG in decoding of imagined kinematics and open new possibilities for future BCI research.

Comparison of Deep Learning and Traditional Machine Learning Classification Performance in a SSVEP Based Brain Computer Interface

Brain-computer interfaces (BCIs) offer a very high potential to help those who cannot use their organs properly. In the literature, many electroencephalogram based BCIs exist. Steady state visual evoked potential (SSVEP) based BCIs provide relatively higher accuracy values which make them very popular in BCI research. Recently, deep learning (DL) based methods have been used in electroencephalogram classification problems and they had superior performance over traditional machine learning (ML) methods, which require feature extraction step. This study aimed at comparing the performance of DL and traditional ML based classification performance in terms of stimuli duration, number of channels, and number of trials in an SSVEP based BCI experiment. In the traditional approach canonical correlation analysis method was used for the feature extraction and then three well-known classifiers were used for classification. In DL-based classification, spatio-spectral decomposition (SSD) method was integrated as a preprocessing step to extract oscillatory signals in the frequency band of interest with a convolutional neural network structure. Obtained offline classification results show that proposed DL approach could generate better accuracy values than traditional ML-based methods for short time segments (< 1 s). Besides, use of SSD as a preprocessing step increased the accuracy of DL classification. Superior performance of proposed SSD based DL approach over the traditional ML methods in short trials shows the feasibility of this approach in future BCI designs. Similar approach can be used in other fields where there are oscillatory activity in the recorded signals.

2020 International brain-computer interface competition: A review.

The brain-computer interface (BCI) has been investigated as a form of communication tool between the brain and external devices. BCIs have been extended beyond communication and control over the years. The 2020 international BCI competition aimed to provide high-quality neuroscientific data for open access that could be used to evaluate the current degree of technical advances in BCI. Although there are a variety of remaining challenges for future BCI advances, we discuss some of more recent application directions: (i) few-shot EEG learning, (ii) micro-sleep detection (iii) imagined speech decoding, (iv) cross-session classification, and (v) EEG(+ear-EEG) detection in an ambulatory environment. Not only did scientists from the BCI field compete, but scholars with a broad variety of backgrounds and nationalities participated in the competition to address these challenges. Each dataset was prepared and separated into three data that were released to the competitors in the form of training and validation sets followed by a test set. Remarkable BCI advances were identified through the 2020 competition and indicated some trends of interest to BCI researchers.

The Application of Electroencephalogram in Driving Safety: Current Status and Future Prospects.

The driver is one of the most important factors in the safety of the transportation system. The driver’s perceptual characteristics are closely related to driving behavior, while electroencephalogram (EEG) as the gold standard for evaluating human perception is non-deceptive. It is essential to study driving characteristics by analyzing the driver’s brain activity pattern, effectively acquiring driver perceptual characteristics, creating a direct connection between the driver’s brain and external devices, and realizing information interchange. This paper first introduces the theories related to EEG, then reviews the applications of EEG in scenarios such as fatigue driving, distracted driving, and emotional driving. The limitations of existing research have been identified and the prospect of EEG application in future brain-computer interface automotive assisted driving systems have been proposed. This review provides guidance for researchers to use EEG to improve driving safety. It also offers valuable suggestions for future research.

Decoding continuous kinetic information of grasp from stereo-electroencephalographic (SEEG) recordings

Objective. Brain–computer interfaces (BCIs) have the potential to bypass damaged neural pathways and restore functionality lost due to injury or disease. Approaches to decoding kinematic information are well documented; however, the decoding of kinetic information has received less attention. Additionally, the possibility of using stereo-electroencephalography (SEEG) for kinetic decoding during hand grasping tasks is still largely unknown. Thus, the objective of this paper is to demonstrate kinetic parameter decoding using SEEG in patients performing a grasping task with two different force levels under two different ascending rates. Approach. Temporal-spectral representations were studied to investigate frequency modulation under different force tasks. Then, force amplitude was decoded from SEEG recordings using multiple decoders, including a linear model, a partial least squares model, an unscented Kalman filter, and three deep learning models (shallow convolutional neural network, deep convolutional neural network and the proposed CNN+RNN neural network). Main results. The current study showed that: (a) for some channel, both low-frequency modulation (event-related desynchronization (ERD)) and high-frequency modulation (event-related synchronization) were sustained during prolonged force holding periods; (b) continuously changing grasp force can be decoded from the SEEG signals; (c) the novel CNN+RNN deep learning model achieved the best decoding performance, with the predicted force magnitude closely aligned to the ground truth under different force amplitudes and changing rates. Significance. This work verified the possibility of decoding continuously changing grasp force using SEEG recordings. The result presented in this study demonstrated the potential of SEEG recordings for future BCI application.

Assessing differential representation of hand movements in multiple domains using stereo-electroencephalographic recordings

Invasive brain-computer interfaces (BCI) have made great progress in the reconstruction of fine hand movement parameters for paralyzed patients, where superficial measurement modalities including electrocorticography (ECoG) and micro-array recordings are mostly used. However, these recording techniques typically focus on the signals from the sensorimotor cortex, leaving subcortical regions and other cortical regions related to the movements largely unexplored. As an intracranial recording technique for the presurgical assessments of brain surgery, stereo-encephalography (SEEG) inserts depth electrodes containing multiple contacts into the brain and thus provides the unique opportunity for investigating movement-related neural representation throughout the brain. Although SEEG samples neural signals with high spatial-temporal resolutions, its potential of being used to build BCIs has just been realized recently, and the decoding of SEEG activity related to hand movements has not been comprehensively investigated yet. Here, we systematically evaluated the factors influencing the performance of movement decoding using SEEG signals recorded from 32 human subjects performing a visually-cued hand movement task. Our results suggest that multiple regions in both lateral and depth directions present significant neural selectivity to the task, whereas the sensorimotor area, including both precentral and postcentral cortex, carries the richest discriminative neural information for the decoding. The posterior parietal and prefrontal cortex contribute gradually less, but still rich sources for extracting movement parameters. The insula, temporal and occipital cortex also contains useful task-related information for decoding. Under the cortex layer, white matter presents decodable neural patterns but yields a lower accuracy (42.0 ± 0.8%) than the cortex on average (44.2 ± 0.8%, p<0.01). Notably, collectively using neural signals from multiple task-related areas can significantly enhance the movement decoding performance by 6.9% (p<0.01) on average compared to using a single region. Among the different spectral components of SEEG activity, the high gamma and delta bands offer the most informative features for hand movements reconstruction. Additionally, the phase-amplitude coupling strength between these two frequency ranges correlates positively with the performance of movement decoding. In the temporal domain, maximum decoding accuracy is first reached around 2 s after the onset of movement commands. In sum, this study provides valuable insights for the future motor BCIs design employing both SEEG recordings and other recording modalities.

A 1D CNN for high accuracy classification and transfer learning in motor imagery EEG-based brain-computer interface

Objective. Brain-computer interface (BCI) aims to establish communication paths between the brain processes and external devices. Different methods have been used to extract human intentions from electroencephalography (EEG) recordings. Those based on motor imagery (MI) seem to have a great potential for future applications. These approaches rely on the extraction of EEG distinctive patterns during imagined movements. Techniques able to extract patterns from raw signals represent an important target for BCI as they do not need labor-intensive data pre-processing. Approach. We propose a new approach based on a 10-layer one-dimensional convolution neural network (1D-CNN) to classify five brain states (four MI classes plus a ‘baseline’ class) using a data augmentation algorithm and a limited number of EEG channels. In addition, we present a transfer learning method used to extract critical features from the EEG group dataset and then to customize the model to the single individual by training its late layers with only 12-min individual-related data. Main results. The model tested with the ‘EEG Motor Movement/Imagery Dataset’ outperforms the current state-of-the-art models by achieving a accuracy at the group level. In addition, the transfer learning approach we present achieves an average accuracy of . Significance. The proposed methods could foster the development of future BCI applications relying on few-channel portable recording devices and individual-based training.

Magnetometers vs Gradiometers for Neural Speech Decoding.

Neural speech decoding aims at providing natural rate communication assistance to patients with locked-in state (e.g. due to amyotrophic lateral sclerosis, ALS) in contrast to the traditional brain-computer interface (BCI) spellers which are slow. Recent studies have shown that Magnetoencephalography (MEG) is a suitable neuroimaging modality to study neural speech decoding considering its excellent temporal resolution that can characterize the fast dynamics of speech. Gradiometers have been the preferred choice for sensor space analysis with MEG, due to their efficacy in noise suppression over magnetometers. However, recent development of optically pumped magnetometers (OPM) based wearable-MEG devices have shown great potential in future BCI applications, yet, no prior study has evaluated the performance of magnetometers in neural speech decoding. In this study, we decoded imagined and spoken speech from the MEG signals of seven healthy participants and compared the performance of magnetometers and gradiometers. Experimental results indicated that magnetometers also have the potential for neural speech decoding, although the performance was significantly lower than that obtained with gradiometers. Further, we implemented a wavelet based denoising strategy that improved the performance of both magnetometers and gradiometers significantly. These findings reconfirm that gradiometers are preferable in MEG based decoding analysis but also provide the possibility towards the use of magnetometers (or OPMs) for the development of the next-generation speech-BCIs.

Analysis and Classification of Speech Imagery EEG Based on Chinese Initials

Brain-computer interfaces (BCI) can provide external information communication for people with normal thinking but impaired motor functions. For patients with language disorders, speech imagery BCIs make it possible to communicate normally. However, there are few studies on Chinese speech imagery at present. Almost all studies employed fixed experimental content, without considering the diversity in subjects. With the purpose of improving the effect of a Chinese speech imagery BCI system, a novel experiment of Chinese initials imagery was designed. The experiment is divided into two parts. A preliminary experiment used to select content for subjects. Formal experiment-specific experimental content was designed for subjects. After preprocessing, feature extraction was carried out by common spatial patterns (CSP) and discrete wavelet transform (DWT), and then a support vector machine (SVM) and extreme learning machine (ELM) were used for classification. Finally, the best performance was obtained by the model using DWT and ELM with a highest accuracy of 73.04%. This study shows that the novel experiment is feasible and can potentially extend the capability of utilizing speech imagery in future BCI applications.

Efficacy of Brain-Computer Interface and the Impact of Its Design Characteristics on Poststroke Upper-limb Rehabilitation: A Systematic Review and Meta-analysis of Randomized Controlled Trials.

Background. A number of recent randomized controlled trials reported the efficacy of brain–computer interface (BCI) for upper-limb stroke rehabilitation compared with other therapies. Despite the encouraging results reported, there is a significant variance in the reported outcomes. This paper aims to investigate the effectiveness of different BCI designs on poststroke upper-limb rehabilitation. Methods. The effect sizes of pooled and individual studies were assessed by computing Hedge’s g values with a 95% confidence interval. Subgroup analyses were also performed to examine the impact of different BCI designs on the treatment effect. Results. The study included 12 clinical trials involving 298 patients. The analysis showed that the BCI yielded significant superior short-term and long-term efficacy in improving the upper-limb motor function compared to the control therapies (Hedge’s g = 0.73 and 0.33, respectively). Based on our subgroup analyses, the BCI studies that used the intention of movement had a higher effect size compared to those used motor imagery (Hedge’s g = 1.21 and 0.55, respectively). The BCI studies using band power features had a significantly higher effect size than those using filter bank common spatial patterns features (Hedge’s g = 1.25 and − 0.23, respectively). Finally, the studies that used functional electrical stimulation as the BCI feedback had the highest effect size compared to other devices (Hedge’s g = 1.2). Conclusion. This meta-analysis confirmed the effectiveness of BCI for upper-limb rehabilitation. Our findings support the use of band power features, the intention of movement, and the functional electrical stimulation in future BCI designs for poststroke upper-limb rehabilitation.

BCI learning induces core-periphery reorganization in M/EEG multiplex brain networks

Objective. Brain–computer interfaces (BCIs) constitute a promising tool for communication and control. However, mastering non-invasive closed-loop systems remains a learned skill that is difficult to develop for a non-negligible proportion of users. The involved learning process induces neural changes associated with a brain network reorganization that remains poorly understood. Approach. To address this inter-subject variability, we adopted a multilayer approach to integrate brain network properties from electroencephalographic and magnetoencephalographic data resulting from a four-session BCI training program followed by a group of healthy subjects. Our method gives access to the contribution of each layer to multilayer network that tends to be equal with time. Main results. We show that regardless the chosen modality, a progressive increase in the integration of somatosensory areas in the α band was paralleled by a decrease of the integration of visual processing and working memory areas in the β band. Notably, only brain network properties in multilayer network correlated with future BCI scores in the α 2 band: positively in somatosensory and decision-making related areas and negatively in associative areas. Significance. Our findings cast new light on neural processes underlying BCI training. Integrating multimodal brain network properties provides new information that correlates with behavioral performance and could be considered as a potential marker of BCI learning.

Extracellular voltage thresholds for maximizing information extraction in primate auditory cortex: implications for a brain computer interface

Objective. Research by Oby (2016 J. Neural. Eng. 13 036009) demonstrated that the optimal threshold for extracting information from visual and motor cortices may differ from the optimal threshold for identifying single neurons via spike sorting methods. The optimal threshold for extracting information from auditory cortex has yet to be identified, nor has the optimal temporal scale for representing auditory cortical activity. Here, we describe a procedure to jointly optimize the extracellular threshold and bin size with respect to the decoding accuracy achieved by a linear classifier for a diverse set of auditory stimuli. Approach. We used linear multichannel arrays to record extracellular neural activity from the auditory cortex of awake squirrel monkeys passively listening to both simple and complex sounds. We executed a grid search of the coordinate space defined by the voltage threshold (in units of standard deviation) and the bin size (in units of milliseconds), and computed decoding accuracy at each point. Main results. The optimal threshold for information extraction was consistently near two standard deviations below the voltage trace mean, which falls significantly below the range of three to five standard deviations typically used as inputs to spike sorting algorithms in basic research and in brain-computer interface (BCI) applications. The optimal binwidth was minimized at the optimal voltage threshold, particularly for acoustic stimuli dominated by temporally dynamic features, indicating that permissive thresholding permits readout of cortical responses with temporal precision on the order of a few milliseconds. Significance. The improvements in decoding accuracy we observed for optimal readout parameters suggest that standard thresholding methods substantially underestimate the information present in auditory cortical spiking patterns. The fact that optimal thresholds were relatively low indicates that local populations of cortical neurons exhibit high temporal coherence that could be leveraged in service of future auditory BCI applications.

Detection of the Intention of Direction Changes During Gait Through EEG Signals.

Brain-Computer Interfaces (BCIs) are becoming an important technological tool for the rehabilitation process of patients with locomotor problems, due to their ability to recover the connection between brain and limbs by promoting neural plasticity. They can be used as assistive devices to improve the mobility of handicapped people. For this reason, current BCIs have to be improved to allow an accurate and natural use of external devices. This work proposes a novel methodology for the detection of the intention to change the direction during gait based on event-related desynchronization (ERD). Frequency and temporal features of the electroencephalographic (EEG) signals are characterized. Then, a selection of the most influential features and electrodes to differentiate the direction change intention from the walking is carried out. Best results are obtained when combining frequency and temporal features with an average accuracy of [Formula: see text]%, which are promising to be applied for future BCIs.

Muscle Artifact Removal Toward Mobile SSVEP-Based BCI: A Comparative Study

Steady-state visual evoked potential (SSVEP) serves as one of the extensively utilized paradigms for brain–computer interface (BCI). SSVEP-based BCI has advantages of good classification accuracy, high information transfer rate (ITR), and little user training. However, there exist few studies investigating SSVEP-based BCI in the mobile situation, where electroencephalography (EEG) data are inevitably contaminated by muscle activities, and meanwhile, a limited number of channels are preferred. In this study, two major muscle artifact removal schemes are investigated in the dynamic scenario. One scheme adopts state-of-the-art blind source separation (BSS) methods without electromyogram (EMG) reference, while the other utilizes auxiliary EMG channels as a reference in the denoising process. To evaluate the performance of two schemes, we have tested them on real EEG data collected from ten subjects. The results demonstrate that, in the few-channel condition of SSVEP data, there are classification accuracy improvements of 9.93% for the canonical correlation analysis (CCA) indicator and 15.72% for the task-related component analysis (TRCA) indicator after processed by recursive least-squares (RLS) filtering with auxiliary EMG, and in the single-channel condition, there are classification accuracy improvements of 22.00% for the CCA indicator and 24.42% for the TRCA indicator after RLS filtering, averaged across all the subjects and data lengths. However, the BSS-based methods fail to improve the classification accuracies of SSVEP. Thus, it is of necessity to mount additional EMG channels for effective muscle activity suppression. This study provides an insightful suggestion for future real-life BCI applications.

Effects of Skin Friction on Tactile P300 Brain-Computer Interface Performance.

Tactile perception, the primary sensing channel of the tactile brain-computer interface (BCI), is a complicated process. Skin friction plays a vital role in tactile perception. This study aimed to examine the effects of skin friction on tactile P300 BCI performance. Two kinds of oddball paradigms were designed, silk-stim paradigm (SSP) and linen-stim paradigm (LSP), in which silk and linen were wrapped on target vibration motors, respectively. In both paradigms, the disturbance vibrators were wrapped in cotton. The experimental results showed that LSP could induce stronger event-related potentials (ERPs) and achieved a higher classification accuracy and information transfer rate (ITR) compared with SSP. The findings indicate that high skin friction can achieve high performance in tactile BCI. This work provides a novel research direction and constitutes a viable basis for the future tactile P300 BCI, which may benefit patients with visual impairments.

P300 latency jitter and its correlates in people with amyotrophic lateral sclerosis

ObjectivePeople with amyotrophic lateral sclerosis (ALS) can benefit from brain-computer interfaces (BCIs). However, users with ALS may experience significant variations in BCI performance and event-related potential (ERP) characteristics. This study investigated latency jitter and its correlates in ALS. MethodsElectroencephalographic (EEG) responses were recorded from six people with ALS and nine neurotypical controls. ERP amplitudes and latencies were extracted. Classifier-based latency estimation was used to calculate latency jitter. ERP components and latency jitter were compared between groups using Wilcoxon rank-sum tests. Correlations between latency jitter and each of the clinical measures, ERP features, and performance measures were investigated using Spearman and repeated measures correlations. ResultsLatency jitter was significantly increased in participants with ALS and significantly negatively correlated with BCI performance in both ALS and control participants. ERP amplitudes were significantly attenuated in ALS, and significant correlations between ERP features and latency jitter were observed. There was no significant correlation between latency jitter and clinical measures. ConclusionsLatency jitter is increased in ALS and correlates with both BCI performance and ERP features. SignificanceThese results highlight the associations of latency jitter with BCI performance and ERP characteristics and could inform future BCI designs for people with ALS.

BCIAUT-P300: A Multi-Session and Multi-Subject Benchmark Dataset on Autism for P300-Based Brain-Computer-Interfaces.

There is a lack of multi-session P300 datasets for Brain-Computer Interfaces (BCI). Publicly available datasets are usually limited by small number of participants with few BCI sessions. In this sense, the lack of large, comprehensive datasets with various individuals and multiple sessions has limited advances in the development of more effective data processing and analysis methods for BCI systems. This is particularly evident to explore the feasibility of deep learning methods that require large datasets. Here we present the BCIAUT-P300 dataset, containing 15 autism spectrum disorder individuals undergoing 7 sessions of P300-based BCI joint-attention training, for a total of 105 sessions. The dataset was used for the 2019 IFMBE Scientific Challenge organized during MEDICON 2019 where, in two phases, teams from all over the world tried to achieve the best possible object-detection accuracy based on the P300 signals. This paper presents the characteristics of the dataset and the approaches followed by the 9 finalist teams during the competition. The winner obtained an average accuracy of 92.3% with a convolutional neural network based on EEGNet. The dataset is now publicly released and stands as a benchmark for future P300-based BCI algorithms based on multiple session data.

Exploration of User's Mental State Changes during Performing Brain-Computer Interface.

Substantial developments have been established in the past few years for enhancing the performance of brain–computer interface (BCI) based on steady-state visual evoked potential (SSVEP). The past SSVEP-BCI studies utilized different target frequencies with flashing stimuli in many different applications. However, it is not easy to recognize user’s mental state changes when performing the SSVEP-BCI task. What we could observe was the increasing EEG power of the target frequency from the user’s visual area. BCI user’s cognitive state changes, especially in mental focus state or lost-in-thought state, will affect the BCI performance in sustained usage of SSVEP. Therefore, how to differentiate BCI users’ physiological state through exploring their neural activities changes while performing SSVEP is a key technology for enhancing the BCI performance. In this study, we designed a new BCI experiment which combined working memory task into the flashing targets of SSVEP task using 12 Hz or 30 Hz frequencies. Through exploring the EEG activity changes corresponding to the working memory and SSVEP task performance, we can recognize if the user’s cognitive state is in mental focus or lost-in-thought. Experiment results show that the delta (1–4 Hz), theta (4–7 Hz), and beta (13–30 Hz) EEG activities increased more in mental focus than in lost-in-thought state at the frontal lobe. In addition, the powers of the delta (1–4 Hz), alpha (8–12 Hz), and beta (13–30 Hz) bands increased more in mental focus in comparison with the lost-in-thought state at the occipital lobe. In addition, the average classification performance across subjects for the KNN and the Bayesian network classifiers were observed as 77% to 80%. These results show how mental state changes affect the performance of BCI users. In this work, we developed a new scenario to recognize the user’s cognitive state during performing BCI tasks. These findings can be used as the novel neural markers in future BCI developments.

Exploring differences between self-report and electrophysiological indices of drowsy driving: A usability examination of a personal brain-computer interface device

IntroductionImpaired driving has resulted in numerous accidents, fatalities, and costly damage. One particularly concerning type of impairment is driver drowsiness. Despite advancements, modern vehicle safety systems remain ineffective at keeping drowsy drivers alert and aware of their state, even temporarily. Until recently the use of user-centric brain-computer interface (BCI) devices to capture electrophysiological data relating to driver drowsiness has been limited. Method: In this study, 25 participants drove on a simulated roadway under drowsy conditions. Results: Neither subjective nor electrophysiological measures differed between individuals who showed overt signs of drowsiness (prolonged eye closure) during the drive. However, the directionality and effect size estimates provided by the BCI device suggested the practicality and feasibility of its future implementation in vehicle safety systems. Practical applications: This research highlights opportunities for future BCI device research for use to assess the state of drowsy drivers in a real-world context.

Brain–Computer Interface Software: A Review and Discussion

Software is a critical component of brain–computer interfaces (BCIs). While BCI hardware enables the retrieval of brain signals, BCI software is required to analyze these signals, produce output, and provide feedback. Users from multiple research areas have adopted BCI software platforms to investigate various concepts. Recently, interest in web-based BCI software has also emerged. The system design and control signal techniques of state-of-the-art BCI software platforms have been previously investigated. However, there is limited literature discussing user adoption of BCI software platforms. Additionally, there is a lack of work discussing the recent emergence of web tools relevant to BCI applications. This article aims to address these gaps by presenting a bibliometric review of the state-of-the-art BCI software. Furthermore, we discuss web-based BCIs and present tools that may be used to develop future web-based BCI applications.