Brain-computer Interface Experiment Research Articles

Correcting for ERP latency jitter improves gaze-independent BCI decoding

Objective. Patients suffering from heavy paralysis or Locked-in-Syndrome can regain communication using a Brain–Computer Interface (BCI). Visual event-related potential (ERP) based BCI paradigms exploit visuospatial attention (VSA) to targets laid out on a screen. However, performance drops if the user does not direct their eye gaze at the intended target, harming the utility of this class of BCIs for patients suffering from eye motor deficits. We aim to create an ERP decoder that is less dependent on eye gaze. Approach. ERP component latency jitter plays a role in covert visuospatial attention (VSA) decoding. We introduce a novel decoder which compensates for these latency effects, termed Woody Classifier-based Latency Estimation (WCBLE). We carried out a BCI experiment recording ERP data in overt and covert visuospatial attention (VSA), and introduce a novel special case of covert VSA termed split VSA, simulating the experience of patients with severely impaired eye motor control. We evaluate WCBLE on this dataset and the BNCI2014-009 dataset, within and across VSA conditions to study the dependency on eye gaze and the variation thereof during the experiment. Main results. WCBLE outperforms state-of-the-art methods in the VSA conditions of interest in gaze-independent decoding, without reducing overt VSA performance. Results from across-condition evaluation show that WCBLE is more robust to varying VSA conditions throughout a BCI operation session. Significance. Together, these results point towards a pathway to achieving gaze independence through suited ERP decoding. Our proposed gaze-independent solution enhances decoding performance in those cases where performing overt VSA is not possible.

Evaluation of an online SSVEP-BCI with fast system setup

The brain–computer interface (BCI) plays an important role in neural restoration. Current BCI systems generally require complex experimental preparation to perform well, but this time-consuming process may hinder their use in clinical applications. To explore the feasibility of simplifying the BCI system setup, a wearable BCI system based on the steady-state visual evoked potential (SSVEP) was developed and evaluated. Fifteen healthy participants were recruited to test the fast-setup system using dry and wet electrodes in a real-life scenario. In this study, the average system setup time for the dry electrode was 38.40 seconds and that for the wet electrode was 103.40 seconds, which are times appreciably shorter than those in previous BCI experiments, enabling a rapid setup of the BCI system. Although the electroencephalogram (EEG) signal quality was low in this fast-setup BCI experiment, the BCI system achieved an information transfer rate of 138.89 bits/min with an eight-channel wet electrode and an information transfer rate of 70.59 bits/min with an eight-channel dry electrode, showing that the overall performance was close to that in traditional experiments. In addition, the results suggest that the solutions of a multi-channel dry electrode or few-channel wet electrode may be suitable for the fast-setup SSEVP-BCI. This fast-setup SSVEP-BCI has the advantages of simple preparation and stable performance and is thus conducive to promoting the use of the BCI in clinical practice.

Real-time estimation of EEG-based engagement in different tasks

Objective. Recent trends in brain–computer interface (BCI) research concern the passive monitoring of brain activity, which aim to monitor a wide variety of cognitive states. Engagement is such a cognitive state, which is of interest in contexts such as learning, entertainment or rehabilitation. This study proposes a novel approach for real-time estimation of engagement during different tasks using electroencephalography (EEG). Approach. Twenty-three healthy subjects participated in the BCI experiment. A modified version of the d2 test was used to elicit engagement. Within-subject classification models which discriminate between engaging and resting states were trained based on EEG recorded during a d2 test based paradigm. The EEG was recorded using eight electrodes and the classification model was based on filter-bank common spatial patterns and a linear discriminant analysis. The classification models were evaluated in cross-task applications, namely when playing Tetris at different speeds (i.e. slow, medium, fast) and when watching two videos (i.e. advertisement and landscape video). Additionally, subjects’ perceived engagement was quantified using a questionnaire. Main results. The models achieved a classification accuracy of 90% on average when tested on an independent d2 test paradigm recording. Subjects’ perceived and estimated engagement were found to be greater during the advertisement compared to the landscape video (p = 0.025 and p < 0.001, respectively); greater during medium and fast compared to slow Tetris speed (p < 0.001, respectively); not different between medium and fast Tetris speeds. Additionally, a common linear relationship was observed for perceived and estimated engagement (r rm = 0.44, p < 0.001). Finally, theta and alpha band powers were investigated, which respectively increased and decreased during more engaging states. Significance. This study proposes a task-specific EEG engagement estimation model with cross-task capabilities, offering a framework for real-world applications.

Detection of common EEG phenomena using individual electrodes placed outside the hair

Many studies over the past decades have provided exciting evidence that electrical signals recorded from the scalp (electroencephalogram, EEG) hold meaningful information about the brain’s function or dysfunction. This information is used routinely in research laboratories to test specific hypotheses and in clinical settings to aid in diagnoses (such as during polysomnography evaluations). Unfortunately, with very few exceptions, such meaningful information about brain function has not yet led to valuable solutions that can address the needs of many people outside such research laboratories or clinics. One of the major hurdles to practical application of EEG-based neurotechnologies is the current predominant requirement to use electrodes that are placed in the hair, which greatly reduces practicality and cosmesis. While several studies reported results using one specific combination of signal/reference electrode outside the hair in one specific context (such as a brain-computer interface experiment), it has been unclear what information about brain function can be acquired using different signal/referencing locations placed outside the hair. To address this issue, in this study, we set out to determine to what extent EEG phenomena related to auditory, visual, cognitive, motor, and sleep function can be detected from different combinations of individual signal/referencing electrodes that are placed outside the hair. The results of our study from 15 subjects suggest that only a few EEG electrodes placed in locations on the forehead or around the ear can provide substantial task-related information in 6 of 7 tasks. Thus, the results of our study provide encouraging evidence and guidance that should invigorate and facilitate the translation of laboratory experiments into practical, useful, and valuable EEG-based neurotechnology solutions.

Deep Learning Approaches for Electroencephalography (EEG)-Based User Response Prediction

In deep learning, finding the best algorithms for time series data can be challenging due to its stochastic and nonlinear nature. This study endeavours to address the challenges posed by a 10-class classification and binary classification problem employing deep learning algorithms. We collected Electroencephalography (EEG) data from participants engaged in the Corsi Block Tapping Task, utilizing various combinations of Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Bidirectional LSTM (BiLSTM) models across multiple layers to achieve the highest accuracy across different frequency bands - namely, beta band (14-30 Hz), alpha band (8-13 Hz), theta band (4-8 Hz), and delta band (0.5-4.35 Hz). Our findings in the context of the 10-class classification problem highlight the superior performance of the 1 CNN + 4 linear layers model, boasting an accuracy of 64.47%. In the realm of binary classification, the 1 LSTM + 4 linear layers model emerged as the top performer, achieving an impressive accuracy of 93.30%. Notably, the beta wave exhibited enhanced predictive capabilities. These results hold promising implications for the design of brain-computer interface experiments, where specific brain regions can predict responses with heightened accuracy. Furthermore, future applications may encompass the development of cognitive systems where both time and accuracy play pivotal roles.

Methods for motion artifact reduction in online brain-computer interface experiments: a systematic review.

Brain-computer interfaces (BCIs) have emerged as a promising technology for enhancing communication between the human brain and external devices. Electroencephalography (EEG) is particularly promising in this regard because it has high temporal resolution and can be easily worn on the head in everyday life. However, motion artifacts caused by muscle activity, fasciculation, cable swings, or magnetic induction pose significant challenges in real-world BCI applications. In this paper, we present a systematic review of methods for motion artifact reduction in online BCI experiments. Using the PRISMA filter method, we conducted a comprehensive literature search on PubMed, focusing on open access publications from 1966 to 2022. We evaluated 2,333 publications based on predefined filtering rules to identify existing methods and pipelines for motion artifact reduction in EEG data. We present a lookup table of all papers that passed the defined filters, all used methods, and pipelines and compare their overall performance and suitability for online BCI experiments. We summarize suitable methods, algorithms, and concepts for motion artifact reduction in online BCI applications, highlight potential research gaps, and discuss existing community consensus. This review aims to provide a comprehensive overview of the current state of the field and guide researchers in selecting appropriate methods for motion artifact reduction in online BCI experiments.

Manifold embedded instance selection to suppress negative transfer in motor imagery-based brain–computer interface

Individual distribution discrepancy poses significant challenges to cross-subject electroencephalography (EEG) signal decoding. Although transfer learning has emerged as an effective approach to minimize this distribution gap, EEG signals’ low voltage amplitude makes them vulnerable to noise and results in abnormal, low-quality samples that cause negative transfer phenomenon. This phenomenon, in turn, undermines the efficacy of transfer learning and impedes brain-computer interface (BCI) applications. To overcome this challenge, we introduce the manifold embedded instance selection (MEIS) algorithm, which addresses negative transfer. The MEIS algorithm operates in two ways: converting raw EEG matrices into manifold embedded vectors that maintain sample discriminability, and designing an evaluator to assess the transferability of samples and filter out negative transfer samples from the source domain. When faced with a large number of source domains, our proposed method employs domain similarity estimation to determine the most beneficial subset of source domains for the target domain. The effectiveness of the proposed method is confirmed via offline and simulated online motor imagery-based BCI experiments where it demonstrates superior accuracy compared to other advanced techniques. Additionally, the MEIS algorithm significantly reduces both training time and the number of required training samples without affecting the model’s performance. The code is available at https://github.com/ZilinL/MEIS.

A large EEG database with users’ profile information for motor imagery brain-computer interface research

We present and share a large database containing electroencephalographic signals from 87 human participants, collected during a single day of brain-computer interface (BCI) experiments, organized into 3 datasets (A, B, and C) that were all recorded using the same protocol: right and left hand motor imagery (MI). Each session contains 240 trials (120 per class), which represents more than 20,800 trials, or approximately 70 hours of recording time. It includes the performance of the associated BCI users, detailed information about the demographics, personality profile as well as some cognitive traits and the experimental instructions and codes (executed in the open-source platform OpenViBE). Such database could prove useful for various studies, including but not limited to: (1) studying the relationships between BCI users’ profiles and their BCI performances, (2) studying how EEG signals properties varies for different users’ profiles and MI tasks, (3) using the large number of participants to design cross-user BCI machine learning algorithms or (4) incorporating users’ profile information into the design of EEG signal classification algorithms.

Window-Adjusted Common Spatial Pattern for Detecting Error-Related Potentials in P300 BCI

Under certain task conditions, error-related potential (ErrP) will be elicited, meaning that the subject is perceiving an error, responding to an external error, or engaging in a cognitive process of reinforcement learning. The detection of ErrP on a single trial basis has been studied and applied to improve all kinds of brain–computer interfaces (BCIs). However, the performance of this kind of detection is not currently good enough. In the paper, we proposed a novel method, called window-adjusted common spatial pattern (WACSP), for detecting ErrP in P300 BCI. In this method, the coefficient of determination was introduced to measure the difference of Electroencephalogram (EEG) signals on a channel at a moment and to guide the search of time windows in which EEG differences are significant, and common spatial pattern (CSP) was further used to capture the stable spatial patterns of EEG differences between correct and incorrect responses in each time window. WACSP and the commonly used methods were tested on the data sets that were built using the EEG signals acquired during the P300 BCI experiments with different feedback. The comparisons of accuracy, area under receiver operating characteristics curve (AUC) and F-measure show that WACSP significantly outperforms the commonly used methods. The proposed method can improve ErrP detection based on a single trial.

Online adaptive group-wise sparse Penalized Recursive Exponentially Weighted N-way Partial Least Square for epidural intracranial BCI.

Motor Brain-Computer Interfaces (BCIs) create new communication pathways between the brain and external effectors for patients with severe motor impairments. Control of complex effectors such as robotic arms or exoskeletons is generally based on the real-time decoding of high-resolution neural signals. However, high-dimensional and noisy brain signals pose challenges, such as limitations in the generalization ability of the decoding model and increased computational demands. The use of sparse decoders may offer a way to address these challenges. A sparsity-promoting penalization is a common approach to obtaining a sparse solution. BCI features are naturally structured and grouped according to spatial (electrodes), frequency, and temporal dimensions. Applying group-wise sparsity, where the coefficients of a group are set to zero simultaneously, has the potential to decrease computational time and memory usage, as well as simplify data transfer. Additionally, online closed-loop decoder adaptation (CLDA) is known to be an efficient procedure for BCI decoder training, taking into account neuronal feedback. In this study, we propose a new algorithm for online closed-loop training of group-wise sparse multilinear decoders using L p -Penalized Recursive Exponentially Weighted N-way Partial Least Square (PREW-NPLS). Three types of sparsity-promoting penalization were explored using L p with p = 0., 0.5, and 1. The algorithms were tested offline in a pseudo-online manner for features grouped by spatial dimension. A comparison study was conducted using an epidural ECoG dataset recorded from a tetraplegic individual during long-term BCI experiments for controlling a virtual avatar (left/right-hand 3D translation). Novel algorithms showed comparable or better decoding performance than conventional REW-NPLS, which was achieved with sparse models. The proposed algorithms are compatible with real-time CLDA. The proposed algorithm demonstrated good performance while drastically reducing the computational load and the memory consumption. However, the current study is limited to offline computation on data recorded with a single patient, with penalization restricted to the spatial domain only.

Single trial detection of error-related potentials in brain–machine interfaces: a survey and comparison of methods

Objective. Error-related potential (ErrP) is a potential elicited in the brain when humans perceive an error. ErrPs have been researched in a variety of contexts, such as to increase the reliability of brain–computer interfaces (BCIs), increase the naturalness of human–machine interaction systems, teach systems, as well as study clinical conditions. Still, there is a significant challenge in detecting ErrP from a single trial, which may hamper its effective use. The literature presents ErrP detection accuracies quite variable across studies, which raises the question of whether this variability depends more on classification pipelines or on the quality of elicited ErrPs (mostly directly related to the underlying paradigms). Approach. With this purpose, 11 datasets have been used to compare several classification pipelines which were selected according to the studies that reported online performance above 75%. We also analyze the effects of different steps of the pipelines, such as resampling, window selection, augmentation, feature extraction, and classification. Main results. From our analysis, we have found that shrinkage-regularized linear discriminant analysis is the most robust method for classification, and for feature extraction, using Fisher criterion beamformer spatial features and overlapped window averages result in better classification performance. The overall experimental results suggest that classification accuracy is highly dependent on user tasks in BCI experiments and on signal quality (in terms of ErrP morphology, signal-to-noise ratio (SNR), and discrimination). Significance. This study contributes to the BCI research field by responding to the need for a guideline that can direct researchers in designing ErrP-based BCI tasks by accelerating the design steps.

MEDUSA©: A novel Python-based software ecosystem to accelerate brain-computer interface and cognitive neuroscience research

Background and objective: Neurotechnologies have great potential to transform our society in ways that are yet to be uncovered. The rate of development in this field has increased significantly in recent years, but there are still barriers that need to be overcome before bringing neurotechnologies to the general public. One of these barriers is the difficulty of performing experiments that require complex software, such as brain-computer interfaces (BCI) or cognitive neuroscience experiments. Current platforms have limitations in terms of functionality and flexibility to meet the needs of researchers, who often need to implement new experimentation settings. This work was aimed to propose a novel software ecosystem, called MEDUSA©, to overcome these limitations.Methods: We followed strict development practices to optimize MEDUSA© for research in BCI and cognitive neuroscience, making special emphasis in the modularity, flexibility and scalability of our solution. Moreover, it was implemented in Python, an open-source programming language that reduces the development cost by taking advantage from its high-level syntax and large number of community packages.Results: MEDUSA© provides a complete suite of signal processing functions, including several deep learning architectures or connectivity analysis, and ready-to-use BCI and neuroscience experiments, making it one of the most complete solutions nowadays. We also put special effort in providing tools to facilitate the development of custom experiments, which can be easily shared with the community through an app market available in our website to promote reproducibility.Conclusions: MEDUSA© is a novel software ecosystem for modern BCI and neurotechnology experimentation that provides state-of-the-art tools and encourages the participation of the community to make a difference for the progress of these fields. Visit the official website at https://www.medusabci.com/ to know more about this project.

Multiple Scale Convolutional Few-Shot Learning Networks for Online P300-Based Brain–Computer Interface and Its Application to Patients With Disorder of Consciousness

P300 brain-computer interfaces (BCIs) have significant potential for detecting and assessing residual consciousness in patients with disorders of consciousness (DoC) but are limited by insufficient data collected from them. In this study, a multiple scale convolutional few-shot learning network (MSCNN-FSL) was proposed to detect and recognize small sample P300 signals. A multiple scale convolutional neural network (MSCNN) was developed to learn different scale features from different scales of receptive fields to obtain more information from electroencephalograms (EEG). Then, a prototypical network with cosine distance was introduced as a classifier to classify and small sample P300 signals. The MSCNN-FSL was evaluated in two independent online BCI experiments. In the first P300 speller experiment, the presented network achieves good character recognition performance with average accuracies of 98.02%±1.70%. In the second experiment, eight healthy controls achieved photo recognition performance with average accuracies of 98.75%±1.49% and three of the twelve DoC patients achieved more than 64% online accuracies with significance. Our results indicated that the proposed MSCNN-FSL could correctly assess the three patients who may have residual consciousness but were misdiagnosed by the Coma Recovery Scale-Revised (CRS-R). Clinical evaluation after two months did prove our BCI assessment results for the three patients.

An Adaptive EEG Classification Algorithm Based on CSSD and ELM_Kernel for Small Training Samples.

Rehabilitation technologies based on brain-computer interface (BCI) have become a promising approach for patients with dyskinesia to regain movement. In BCI experiment, there is often a necessary stage of calibration measurement before the feedback applications. To reduce the time required for initial training, it is of great importance to have a method which can learn to classify electroencephalogram (EEG) signals with a little amount of training data. In this paper, the novel combination of feature extraction and classification algorithm is proposed for classification of EEG signals with a small number of training samples. For feature extraction, the motor imagery EEG signals are pre-processed, and a relative distance criterion is defined to select the optimal combination of channels. Subsequently, common spatial subspace decomposition (CSSD) algorithm and extreme learning machine with kernel (ELM_Kernel) algorithm are used to perform the types of tasks classification of motor imagery EEG signals. Simulation results demonstrate that the proposed method produces a high average classification accuracy of 99.1% on BCI Competition III dataset IVa and 76.92% on BCI Competition IV dataset IIa outperforming state-of-the-art algorithms and obtains a good classification accuracy.

Sensorimotor Rhythm-Based Brain-Computer Interfaces for Motor Tasks Used in Hand Upper Extremity Rehabilitation after Stroke: A Systematic Review.

Brain-computer interfaces (BCIs) are becoming more popular in the neurological rehabilitation field, and sensorimotor rhythm (SMR) is a type of brain oscillation rhythm that can be captured and analyzed in BCIs. Previous reviews have testified to the efficacy of the BCIs, but seldom have they discussed the motor task adopted in BCIs experiments in detail, as well as whether the feedback is suitable for them. We focused on the motor tasks adopted in SMR-based BCIs, as well as the corresponding feedback, and searched articles in PubMed, Embase, Cochrane library, Web of Science, and Scopus and found 442 articles. After a series of screenings, 15 randomized controlled studies were eligible for analysis. We found motor imagery (MI) or motor attempt (MA) are common experimental paradigms in EEG-based BCIs trials. Imagining/attempting to grasp and extend the fingers is the most common, and there were multi-joint movements, including wrist, elbow, and shoulder. There were various types of feedback in MI or MA tasks for hand grasping and extension. Proprioception was used more frequently in a variety of forms. Orthosis, robot, exoskeleton, and functional electrical stimulation can assist the paretic limb movement, and visual feedback can be used as primary feedback or combined forms. However, during the recovery process, there are many bottleneck problems for hand recovery, such as flaccid paralysis or opening the fingers. In practice, we should mainly focus on patients' difficulties, and design one or more motor tasks for patients, with the assistance of the robot, FES, or other combined feedback, to help them to complete a grasp, finger extension, thumb opposition, or other motion. Future research should focus on neurophysiological changes and functional improvements and further elaboration on the changes in neurophysiology during the recovery of motor function.

Biases in BCI experiments: Do we really need to balance stimulus properties across categories?

Brain Computer Interfaces (BCIs) consist of an interaction between humans and computers with a specific mean of communication, such as voice, gestures, or even brain signals that are usually recorded by an Electroencephalogram (EEG). To ensure an optimal interaction, the BCI algorithm typically involves the classification of the input signals into predefined task-specific categories. However, a recurrent problem is that the classifier can easily be biased by uncontrolled experimental conditions, namely covariates, that are unbalanced across the categories. This issue led to the current solution of forcing the balance of these covariates across the different categories which is time consuming and drastically decreases the dataset diversity. The purpose of this research is to evaluate the need for this forced balance in BCI experiments involving EEG data. A typical design of neural BCIs involves repeated experimental trials using visual stimuli to trigger the so-called Event-Related Potential (ERP). The classifier is expected to learn spatio-temporal patterns specific to categories rather than patterns related to uncontrolled stimulus properties, such as psycho-linguistic variables (e.g., phoneme number, familiarity, and age of acquisition) and image properties (e.g., contrast, compactness, and homogeneity). The challenges are then to know how biased the decision is, which features affect the classification the most, which part of the signal is impacted, and what is the probability to perform neural categorization per se. To address these problems, this research has two main objectives: (1) modeling and quantifying the covariate effects to identify spatio-temporal regions of the EEG allowing maximal classification performance while minimizing the biasing effect, and (2) evaluating the need to balance the covariates across categories when studying brain mechanisms. To solve the modeling problem, we propose using a linear parametric analysis applied to some observable and commonly studied covariates to them. The biasing effect is quantified by comparing the regions highly influenced by the covariates with the regions of high categorical contrast, i.e., parts of the ERP allowing a reliable classification. The need to balance the stimulus's inner properties across categories is evaluated by assessing the separability between category-related and covariate-related evoked responses. The procedure is applied to a visual priming experiment where the images represent items belonging to living or non-living entities. The observed covariates are the commonly controlled psycho-linguistic variables and some visual features of the images. As a result, we identified that the category of the stimulus mostly affects the late evoked response. The covariates, when not modeled, have a biasing effect on the classification, essentially in the early evoked response. This effect increases with the diversity of the dataset and the complexity of the algorithm used. As the effects of both psycho-linguistic variables and image features appear outside of the spatio-temporal regions of significant categorical contrast, the proper selection of the region of interest makes the classification reliable. Having proved that the covariate effects can be separated from the categorical effect, our framework can be further used to isolate the category-dependent evoked response from the rest of the EEG to study neural processes involved when seeing living vs. non-living entities.

Motor-Imagery-Based Brain-Computer Interface Using Signal Derivation and Aggregation Functions.

Brain-computer interface (BCI) technologies are popular methods of communication between the human brain and external devices. One of the most popular approaches to BCI is motor imagery (MI). In BCI applications, the electroencephalography (EEG) is a very popular measurement for brain dynamics because of its noninvasive nature. Although there is a high interest in the BCI topic, the performance of existing systems is still far from ideal, due to the difficulty of performing pattern recognition tasks in EEG signals. This difficulty lies in the selection of the correct EEG channels, the signal-to-noise ratio of these signals, and how to discern the redundant information among them. BCI systems are composed of a wide range of components that perform signal preprocessing, feature extraction, and decision making. In this article, we define a new BCI framework, called enhanced fusion framework, where we propose three different ideas to improve the existing MI-based BCI frameworks. First, we include an additional preprocessing step of the signal: a differentiation of the EEG signal that makes it time invariant. Second, we add an additional frequency band as a feature for the system: the sensorimotor rhythm band, and we show its effect on the performance of the system. Finally, we make a profound study of how to make the final decision in the system. We propose the usage of both up to six types of different classifiers and a wide range of aggregation functions (including classical aggregations, Choquet and Sugeno integrals, and their extensions and overlap functions) to fuse the information given by the considered classifiers. We have tested this new system on a dataset of 20 volunteers performing MI-based brain-computer interface experiments. On this dataset, the new system achieved 88.80% accuracy. We also propose an optimized version of our system that is able to obtain up to 90.76%. Furthermore, we find that the pair Choquet/Sugeno integrals and overlap functions are the ones providing the best results.

Comparison of Deep Learning and Traditional Machine Learning Classification Performance in a SSVEP Based Brain Computer Interface

Brain-computer interfaces (BCIs) offer a very high potential to help those who cannot use their organs properly. In the literature, many electroencephalogram based BCIs exist. Steady state visual evoked potential (SSVEP) based BCIs provide relatively higher accuracy values which make them very popular in BCI research. Recently, deep learning (DL) based methods have been used in electroencephalogram classification problems and they had superior performance over traditional machine learning (ML) methods, which require feature extraction step. This study aimed at comparing the performance of DL and traditional ML based classification performance in terms of stimuli duration, number of channels, and number of trials in an SSVEP based BCI experiment. In the traditional approach canonical correlation analysis method was used for the feature extraction and then three well-known classifiers were used for classification. In DL-based classification, spatio-spectral decomposition (SSD) method was integrated as a preprocessing step to extract oscillatory signals in the frequency band of interest with a convolutional neural network structure. Obtained offline classification results show that proposed DL approach could generate better accuracy values than traditional ML-based methods for short time segments (&amp;lt; 1 s). Besides, use of SSD as a preprocessing step increased the accuracy of DL classification. Superior performance of proposed SSD based DL approach over the traditional ML methods in short trials shows the feasibility of this approach in future BCI designs. Similar approach can be used in other fields where there are oscillatory activity in the recorded signals.

Riemannian geometry-based transfer learning for reducing training time in c-VEP BCIs

One of the main problems that a brain-computer interface (BCI) face is that a training stage is required for acquiring training data to calibrate its classification model just before every use. Transfer learning is a promising method for addressing the problem. In this paper, we propose a Riemannian geometry-based transfer learning algorithm for code modulated visual evoked potential (c-VEP)-based BCIs, which can effectively reduce the calibration time without sacrificing the classification accuracy. The algorithm includes the main procedures of log-Euclidean data alignment (LEDA), super-trial construction, covariance matrix estimation, training accuracy-based subject selection (TSS) and minimum distance to mean classification. Among them, the LEDA reduces the difference in data distribution between subjects, whereas the TSS promotes the similarity between a target subject and the source subjects. The resulting performance of transfer learning is improved significantly. Sixteen subjects participated in a c-VEP BCI experiment and the recorded data were used in offline analysis. Leave-one subject-out (LOSO) cross-validation was used to evaluate the proposed algorithm on the data set. The results showed that the algorithm achieved much higher classification accuracy than the subject-specific (baseline) algorithm with the same number of training trials. Equivalently, the algorithm reduces the training time of the BCI at the same performance level and thus facilitates its application in real world.

An adaptive closed-loop ECoG decoder for long-term and stable bimanual control of an exoskeleton by a tetraplegic

Objective. The article aims at addressing 2 challenges to step motor brain-computer interface (BCI) out of laboratories: asynchronous control of complex bimanual effectors with large numbers of degrees of freedom, using chronic and safe recorders, and the decoding performance stability over time without frequent decoder recalibration. Approach. Closed-loop adaptive/incremental decoder training is one strategy to create a model stable over time. Adaptive decoders update their parameters with new incoming data, optimizing the model parameters in real time. It allows cross-session training with multiple recording conditions during closed loop BCI experiments. In the article, an adaptive tensor-based recursive exponentially weighted Markov-switching multi-linear model (REW-MSLM) decoder is proposed. REW-MSLM uses a mixture of expert (ME) architecture, mixing or switching independent decoders (experts) according to the probability estimated by a ‘gating’ model. A Hidden Markov model approach is employed as gating model to improve the decoding robustness and to provide strong idle state support. The ME architecture fits the multi-limb paradigm associating an expert to a particular limb or action. Main results. Asynchronous control of an exoskeleton by a tetraplegic patient using a chronically implanted epidural electrocorticography (EpiCoG) recorder is reported. The stable over a period of six months (without decoder recalibration) eight-dimensional alternative bimanual control of the exoskeleton and its virtual avatar is demonstrated. Significance. Based on the long-term (>36 months) chronic bilateral EpiCoG recordings in a tetraplegic (ClinicalTrials.gov, NCT02550522), we addressed the poorly explored field of asynchronous bimanual BCI. The new decoder was designed to meet to several challenges: the high-dimensional control of a complex effector in experiments closer to real-world behavior (point-to-point pursuit versus conventional center-out tasks), with the ability of the BCI system to act as a stand-alone device switching between idle and control states, and a stable performance over a long period of time without decoder recalibration.