Brain-computer Interface Task Research Articles

A continuous pursuit dataset for online deep learning-based EEG brain-computer interface

This dataset is from an EEG brain-computer interface (BCI) study investigating the use of deep learning (DL) for online continuous pursuit (CP) BCI. In this task, subjects use Motor Imagery (MI) to control a cursor to follow a randomly moving target, instead of a single stationary target used in other traditional BCI tasks. DL methods have recently achieved promising performance in traditional BCI tasks, but most studies investigate offline data analysis using DL algorithms. This dataset consists of ~168 hours of EEG recordings from complex CP BCI experiments, collected from 28 unique human subjects over multiple sessions each, with an online DL-based decoder. The large amount of subject specific data from multiple sessions may be useful for developing new BCI decoders, especially DL methods that require large amounts of training data. By providing this dataset to the public, we hope to help facilitate the development of new or improved BCI decoding algorithms for the complex CP paradigm for continuous object control, bringing EEG-based BCIs closer to real-world applications.

IpsiHand Brain-Computer Interface Therapy Induces Broad Upper Extremity Motor Rehabilitation in Chronic Stroke.

Chronic hemiparetic stroke patients have very limited benefits from current therapies. Brain-computer interface (BCI) engaging the unaffected hemisphere has emerged as a promising novel therapeutic approach for chronic stroke rehabilitation. This study investigated the effectiveness of contralesionally-controlled BCI therapy in chronic stroke patients with impaired upper extremity motor function. We further explored neurophysiological features of motor recovery driven by BCI. We hypothesized that BCI therapy would induce a broad motor recovery in the upper extremity, and there would be corresponding changes in baseline theta and gamma oscillations, which have been shown to be associated with motor recovery. Twenty-six prospectively enrolled chronic hemiparetic stroke patients performed a therapeutic BCI task for 12 weeks. Motor function assessment data and resting state electroencephalogram signals were acquired before initiating BCI therapy and across BCI therapy sessions. The Upper Extremity Fugl-Meyer assessment served as a primary motor outcome assessment tool. Theta-gamma cross-frequency coupling (CFC) was computed and correlated with motor recovery. Chronic stroke patients achieved significant motor improvement in both proximal and distal upper extremity with BCI therapy. Motor function improvement was independent of Botox application. Theta-gamma CFC enhanced bilaterally over the C3/C4 motor electrodes and positively correlated with motor recovery across BCI therapy sessions. BCI therapy resulted in significant motor function improvement across the proximal and distal upper extremities of patients, which significantly correlated with theta-gamma CFC increases in the motor regions. This may represent rhythm-specific cortical oscillatory mechanism for BCI-driven rehabilitation in chronic stroke patients. Advarra Study: https://classic.clinicaltrials.gov/ct2/show/NCT04338971 and Washington University Study: https://classic.clinicaltrials.gov/ct2/show/NCT03611855.

A MultiModal Vigilance (MMV) dataset during RSVP and SSVEP brain-computer interface tasks

Vigilance represents an ability to sustain prolonged attention and plays a crucial role in ensuring the reliability and optimal performance of various tasks. In this report, we describe a MultiModal Vigilance (MMV) dataset comprising seven physiological signals acquired during two Brain-Computer Interface (BCI) tasks. The BCI tasks encompass a rapid serial visual presentation (RSVP)-based target image retrieval task and a steady-state visual evoked potential (SSVEP)-based cursor-control task. The MMV dataset includes four sessions of seven physiological signals for 18 subjects, which encompasses electroencephalogram(EEG), electrooculogram (EOG), electrocardiogram (ECG), photoplethysmogram (PPG), electrodermal activity (EDA), electromyogram (EMG), and eye movement. The MMV dataset provides data from four stages: 1) raw data, 2) pre-processed data, 3) trial data, and 4) feature data that can be directly used for vigilance estimation. We believe this dataset will achieve flexible reuse and meet the various needs of researchers. And this dataset will greatly contribute to advancing research on physiological signal-based vigilance research and estimation.

Brain-computer interface on wrist training with or without neurofeedback in subacute stroke: a study protocol for a double-blinded, randomized control pilot trial.

After a stroke, damage to the part of the brain that controls movement results in the loss of motor function. Brain-computer interface (BCI)-based stroke rehabilitation involves patients imagining movement without physically moving while the system measures the perceptual-motor rhythm in the motor cortex. Visual feedback through virtual reality and functional electrical stimulation is provided simultaneously. The superiority of real BCI over sham BCI in the subacute phase of stroke remains unclear. Therefore, we aim to compare the effects of real and sham BCI on motor function and brain activity among patients with subacute stroke with weak wrist extensor strength. This is a double-blinded randomized controlled trial. Patients with stroke will be categorized into real BCI and sham BCI groups. The BCI task involves wrist extension for 60 min/day, 5 times/week for 4 weeks. Twenty sessions will be conducted. The evaluation will be conducted four times, as follows: before the intervention, 2 weeks after the start of the intervention, immediately after the intervention, and 4 weeks after the intervention. The assessments include a clinical evaluation, electroencephalography, and electromyography using motor-evoked potentials. Patients will be categorized into two groups, as follows: those who will be receiving neurofeedback and those who will not receive this feedback during the BCI rehabilitation training. We will examine the importance of motor imaging feedback, and the effect of patients' continuous participation in the training rather than their being passive.Clinical Trial Registration: KCT0008589.

A Method to Extract Task-Related EEG Feature Based on Lightweight Convolutional Neural Network.

Unlocking task-related EEG spectra is crucial for neuroscience. Traditional convolutional neural networks (CNNs) effectively extract these features but face limitations like overfitting due to small datasets. To address this issue, we propose a lightweight CNN and assess its interpretability through the fully connected layer (FCL). Initially tested with two tasks (Task 1: open vs closed eyes, Task 2: interictal vs ictal stage), the CNN demonstrated enhanced spectral features in the alpha band for Task 1 and the theta band for Task 2, aligning with established neurophysiological characteristics. Subsequent experiments on two brain-computer interface tasks revealed a correlation between delta activity (around 1.55 Hz) and hand movement, with consistent results across pericentral electroencephalogram (EEG) channels. Compared to recent research, our method stands out by delivering task-related spectral features through FCL, resulting in significantly fewer trainable parameters while maintaining comparable interpretability. This indicates its potential suitability for a wider array of EEG decoding scenarios.

EEGGAN-Net: enhancing EEG signal classification through data augmentation.

Emerging brain-computer interface (BCI) technology holds promising potential to enhance the quality of life for individuals with disabilities. Nevertheless, the constrained accuracy of electroencephalography (EEG) signal classification poses numerous hurdles in real-world applications. In response to this predicament, we introduce a novel EEG signal classification model termed EEGGAN-Net, leveraging a data augmentation framework. By incorporating Conditional Generative Adversarial Network (CGAN) data augmentation, a cropped training strategy and a Squeeze-and-Excitation (SE) attention mechanism, EEGGAN-Net adeptly assimilates crucial features from the data, consequently enhancing classification efficacy across diverse BCI tasks. The EEGGAN-Net model exhibits notable performance metrics on the BCI Competition IV-2a and IV-2b datasets. Specifically, it achieves a classification accuracy of 81.3% with a kappa value of 0.751 on the IV-2a dataset, and a classification accuracy of 90.3% with a kappa value of 0.79 on the IV-2b dataset. Remarkably, these results surpass those of four other CNN-based decoding models. In conclusion, the amalgamation of data augmentation and attention mechanisms proves instrumental in acquiring generalized features from EEG signals, ultimately elevating the overall proficiency of EEG signal classification.

Assessing robustness to adversarial attacks in attention-based networks: Case of EEG-based motor imagery classification

The classification of motor imagery (MI) using Electroencephalography (EEG) plays a pivotal role in facilitating communication for individuals with physical limitations through Brain-Computer Interface (BCI) systems. Recent strides in Attention-Based Networks (ATN) have showcased remarkable performance in EEG signal classification, presenting a promising alternative to conventional Convolutional Neural Networks (CNNs). However, while CNNs have been extensively analyzed for their resilience against adversarial attacks, the susceptibility of ATNs in comparable scenarios remains largely unexplored. This paper aims to fill this gap by investigating the robustness of ATNs in adversarial contexts. We propose a high-performing attention-based deep learning model specifically designed for classifying Motor Imagery (MI) brain signals extracted from EEG data. Subsequently, we conduct a thorough series of experiments to assess various attack strategies targeting ATNs employed in EEG-based BCI tasks. Our analysis utilizes the widely recognized BCI Competition 2a dataset to demonstrate the effectiveness of attention mechanisms in BCI endeavors. Despite achieving commendable classification results in terms of accuracy (87.15%) and kappa score (0.8287), our findings reveal the vulnerability of attention-based models to adversarial manipulations (accuracy: 9.07%, kappa score: -0.21), highlighting the imperative for bolstering the robustness of attention architectures for EEG classification tasks.

Continuous tracking using deep learning-based decoding for noninvasive brain-computer interface.

Brain-computer interfaces (BCI) using electroencephalography provide a noninvasive method for users to interact with external devices without the need for muscle activation. While noninvasive BCIs have the potential to improve the quality of lives of healthy and motor-impaired individuals, they currently have limited applications due to inconsistent performance and low degrees of freedom. In this study, we use deep learning (DL)-based decoders for online continuous pursuit (CP), a complex BCI task requiring the user to track an object in 2D space. We developed a labeling system to use CP data for supervised learning, trained DL-based decoders based on two architectures, including a newly proposed adaptation of the PointNet architecture, and evaluated the performance over several online sessions. We rigorously evaluated the DL-based decoders in a total of 28 human participants, and found that the DL-based models improved throughout the sessions as more training data became available and significantly outperformed a traditional BCI decoder by the last session. We also performed additional experiments to test an implementation of transfer learning by pretraining models on data from other subjects, and midsession training to reduce intersession variability. The results from these experiments showed that pretraining did not significantly improve performance, but updating the models' midsession may have some benefit. Overall, these findings support the use of DL-based decoders for improving BCI performance in complex tasks like CP, which can expand the potential applications of BCI devices and help to improve the quality of lives of healthy and motor-impaired individuals.

Adaptation and learning as strategies to maximize reward in neurofeedback tasks.

Adaptation and learning have been observed to contribute to the acquisition of new motor skills and are used as strategies to cope with changing environments. However, it is hard to determine the relative contribution of each when executing goal directed motor tasks. This study explores the dynamics of neural activity during a center-out reaching task with continuous visual feedback under the influence of rotational perturbations. Results for a brain-computer interface (BCI) task performed by two non-human primate (NHP) subjects are compared to simulations from a reinforcement learning agent performing an analogous task. We characterized baseline activity and compared it to the activity after rotational perturbations of different magnitudes were introduced. We employed principal component analysis (PCA) to analyze the spiking activity driving the cursor in the NHP BCI task as well as the activation of the neural network of the reinforcement learning agent. Our analyses reveal that both for the NHPs and the reinforcement learning agent, the task-relevant neural manifold is isomorphic with the task. However, for the NHPs the manifold is largely preserved for all rotational perturbations explored and adaptation of neural activity occurs within this manifold as rotations are compensated by reassignment of regions of the neural space in an angular pattern that cancels said rotations. In contrast, retraining the reinforcement learning agent to reach the targets after rotation results in substantial modifications of the underlying neural manifold. Our findings demonstrate that NHPs adapt their existing neural dynamic repertoire in a quantitatively precise manner to account for perturbations of different magnitudes and they do so in a way that obviates the need for extensive learning.

Aggregating intrinsic information to enhance BCI performance through federated learning

Insufficient data is a long-standing challenge for Brain–Computer Interface (BCI) to build a high-performance deep learning model. Though numerous research groups and institutes collect a multitude of EEG datasets for the same BCI task, sharing EEG data from multiple sites is still challenging due to the heterogeneity of devices. The significance of this challenge cannot be overstated, given the critical role of data diversity in fostering model robustness. However, existing works rarely discuss this issue, predominantly centering their attention on model training within a single dataset, often in the context of inter-subject or inter-session settings. In this work, we propose a hierarchical personalized Federated Learning EEG decoding (FLEEG) framework to surmount this challenge. This innovative framework heralds a new learning paradigm for BCI, enabling datasets with disparate data formats to collaborate in the model training process. Each client is assigned a specific dataset and trains a hierarchical personalized model to manage diverse data formats and facilitate information exchange. Meanwhile, the server coordinates the training procedure to harness knowledge gleaned from all datasets, thus elevating overall performance. The framework has been evaluated in Motor Imagery (MI) classification with nine EEG datasets collected by different devices but implementing the same MI task. Results demonstrate that the proposed framework can boost classification performance up to 8.4% by enabling knowledge sharing between multiple datasets, especially for smaller datasets. Visualization results also indicate that the proposed framework can empower the local models to put a stable focus on task-related areas, yielding better performance. To the best of our knowledge, this is the first end-to-end solution to address this important challenge.

Motor imagery EEG task recognition using a nonlinear Granger causality feature extraction and an improved Salp swarm feature selection

In the study of motor imagery (MI) brain-computer interfaces (BCIs), how to improve task classification accuracy has been always one of major challenges in the applications of MI-BCIs. As a type of crucial temporal and spatial feature, nonlinear Granger Causality (NGC) analysis was applied to feature extraction of MI-electroencephalogram (EEG) signals because the constructed brain network features can reflect the causal relationship between different channels in various brain regions. However, the MI-BCI task recognition often suffer from the information redundancy of NGC features, and these redundant features will increase the complexity of the machine learning models and accordingly reduce the prediction accuracy of the classification algorithms. To address this problem, this paper proposes a step-by-step tent chaos simulated annealing salp swarm feature selection (STCSA_SaSFS) algorithm to select an optimal set of features in a wrapper feature selection model. Then, the effectiveness of this feature selection method is verified using a support vector machine (SVM) classifier. Through the study of task related MI-BCI EEG data from ten subjects, the experiments showed that the highest classification accuracy of NGC feature extraction plus STCSA_SaSFS reached 97.19%, and the average classification accuracy was 89.57%. This average classification accuracy was 20.07% higher than that of NGC feature extraction without any feature selection, and it is also 2.96% higher than that of NGC feature extraction plus a traditional SaSFS algorithm. The effectiveness of STCSA_SaSFS was also compared with that of other smart swarm optimization algorithms, such as the sparrow search feature selection algorithm (SpSFS). STCSA_SaSFS outperforms SpSFS with an average classification accuracy of 8.07%. The algorithm was validated using a public dataset validation consisting of 10 subjects, which ultimately showed that the feature selection method proposed in this paper (STSA_SaSAFS) has a large advantage in the classification performance of motor imagery brain-computer interface tasks.

Investigating EEG-based cross-session and cross-task vigilance estimation in BCI systems

Objective. The state of vigilance is crucial for effective performance in brain-computer interface (BCI) tasks, and therefore, it is essential to investigate vigilance levels in BCI tasks. Despite this, most studies have focused on vigilance levels in driving tasks rather than on BCI tasks, and the electroencephalogram (EEG) patterns of vigilance states in different BCI tasks remain unclear. This study aimed to identify similarities and differences in EEG patterns and performances of vigilance estimation in different BCI tasks and sessions. Approach. To achieve this, we built a steady-state visual evoked potential-based BCI system and a rapid serial visual presentation-based BCI system and recruited 18 participants to carry out four BCI experimental sessions over four days. Main results. Our findings demonstrate that specific neural patterns for high and low vigilance levels are relatively stable across sessions. Differential entropy features significantly differ between different vigilance levels in all frequency bands and between BCI tasks in the delta and theta frequency bands, with the theta frequency band features playing a critical role in vigilance estimation. Additionally, prefrontal, temporal, and occipital regions are more relevant to the vigilance state in BCI tasks. Our results suggest that cross-session vigilance estimation is more accurate than cross-task estimation. Significance. Our study clarifies the underlying mechanisms of vigilance state in two BCI tasks and provides a foundation for further research in vigilance estimation in BCI applications.

An improved model using convolutional sliding window-attention network for motor imagery EEG classification.

The classification model of motor imagery-based electroencephalogram (MI-EEG) is a new human-computer interface pattern and a new neural rehabilitation assessment method for diseases such as Parkinson's and stroke. However, existing MI-EEG models often suffer from insufficient richness of spatiotemporal feature extraction, learning ability, and dynamic selection ability. To solve these problems, this work proposed a convolutional sliding window-attention network (CSANet) model composed of novel spatiotemporal convolution, sliding window, and two-stage attention blocks. The model outperformed existing state-of-the-art (SOTA) models in within- and between-individual classification tasks on commonly used MI-EEG datasets BCI-2a and Physionet MI-EEG, with classification accuracies improved by 4.22 and 2.02%, respectively. The experimental results also demonstrated that the proposed type token, sliding window, and local and global multi-head self-attention mechanisms can significantly improve the model's ability to construct, learn, and adaptively select multi-scale spatiotemporal features in MI-EEG signals, and accurately identify electroencephalogram signals in the unilateral motor area. This work provided a novel and accurate classification model for MI-EEG brain-computer interface tasks and proposed a feasible neural rehabilitation assessment scheme based on the model, which could promote the further development and application of MI-EEG methods in neural rehabilitation.

Transformer-Based Network with Optimization for Cross-Subject Motor Imagery Identification.

Exploring the effective signal features of electroencephalogram (EEG) signals is an important issue in the research of brain-computer interface (BCI), and the results can reveal the motor intentions that trigger electrical changes in the brain, which has broad research prospects for feature extraction from EEG data. In contrast to previous EEG decoding methods that are based solely on a convolutional neural network, the traditional convolutional classification algorithm is optimized by combining a transformer mechanism with a constructed end-to-end EEG signal decoding algorithm based on swarm intelligence theory and virtual adversarial training. The use of a self-attention mechanism is studied to expand the receptive field of EEG signals to global dependence and train the neural network by optimizing the global parameters in the model. The proposed model is evaluated on a real-world public dataset and achieves the highest average accuracy of 63.56% in cross-subject experiments, which is significantly higher than that found for recently published algorithms. Additionally, good performance is achieved in decoding motor intentions. The experimental results show that the proposed classification framework promotes the global connection and optimization of EEG signals, which can be further applied to other BCI tasks.

Multi-classification for EEG motor imagery signals using data evaluation-based auto-selected regularized FBCSP and convolutional neural network

In recent years, there has been a renewal of interest in brain–computer interface (BCI). One of the BCI tasks is to classify the EEG motor imagery (MI). A great deal of effort has been made on MI classification. What seems to be lacking, however, is multiple MI classification. This paper develops a single-channel-based convolutional neural network to tackle multi-classification motor imagery tasks. For multi-classification, a single-channel learning strategy can extract effective information from each independent channel, making the information between adjacent channels not affect each other. A data evaluation method and a mutual information-based regularization parameters auto-selection algorithm are also proposed to generate effective spatial filters. The proposed method can be used to tackle the problem of an inaccurate mixed covariance matrix caused by fixed regularization parameters and invalid training data. To illustrate the merits of the proposed methods, we used the tenfold cross-validation accuracy and kappa as the evaluation measures to test two data sets. BCI4-2a and BCI3a data sets have four mental classes. For the BCI4-2a data set, the average accuracy is 79.01%, and the kappa is 0.7202 using data evaluation-based auto-selected filter bank regularized common spatial pattern voting (D-ACSP-V) and single-channel series convolutional neural network (SCS-CNN). Compared to traditional FBRCSP, the proposed method improved accuracy by 7.14% for the BCI4-2a data set. By using the BCI3a data set, the proposed method improved accuracy by 9.54% compared with traditional FBRCSP, the average accuracy of the proposed method is 83.70%, and the kappa is 0.7827.

Single trial detection of error-related potentials in brain–machine interfaces: a survey and comparison of methods

Objective. Error-related potential (ErrP) is a potential elicited in the brain when humans perceive an error. ErrPs have been researched in a variety of contexts, such as to increase the reliability of brain–computer interfaces (BCIs), increase the naturalness of human–machine interaction systems, teach systems, as well as study clinical conditions. Still, there is a significant challenge in detecting ErrP from a single trial, which may hamper its effective use. The literature presents ErrP detection accuracies quite variable across studies, which raises the question of whether this variability depends more on classification pipelines or on the quality of elicited ErrPs (mostly directly related to the underlying paradigms). Approach. With this purpose, 11 datasets have been used to compare several classification pipelines which were selected according to the studies that reported online performance above 75%. We also analyze the effects of different steps of the pipelines, such as resampling, window selection, augmentation, feature extraction, and classification. Main results. From our analysis, we have found that shrinkage-regularized linear discriminant analysis is the most robust method for classification, and for feature extraction, using Fisher criterion beamformer spatial features and overlapped window averages result in better classification performance. The overall experimental results suggest that classification accuracy is highly dependent on user tasks in BCI experiments and on signal quality (in terms of ErrP morphology, signal-to-noise ratio (SNR), and discrimination). Significance. This study contributes to the BCI research field by responding to the need for a guideline that can direct researchers in designing ErrP-based BCI tasks by accelerating the design steps.

Less is More: Brain Functional Connectivity Empowered Generalisable Intention Classification with Task-relevant Channel Selection.

Electroencephalography (EEG) signals are gaining popularity in Brain-Computer Interface (BCI)-based rehabilitation and neural engineering applications thanks to their portability and availability. Inevitably, the sensory electrodes on the entire scalp would collect signals irrelevant to the particular BCI task, increasing the risks of overfitting in machine learning-based predictions. While this issue is being addressed by scaling up the EEG datasets and handcrafting the complex predictive models, this also leads to increased computation costs. Moreover, the model trained for one set of subjects cannot easily be adapted to other sets due to inter-subject variability, which creates even higher over-fitting risks. Meanwhile, despite previous studies using either convolutional neural networks (CNNs) or graph neural networks (GNNs) to determine spatial correlations between brain regions, they fail to capture brain functional connectivity beyond physical proximity. To this end, we propose 1) removing task-irrelevant noises instead of merely complicating models; 2) extracting subject-invariant discriminative EEG encodings, by taking functional connectivity into account. Specifically, we construct a task-adaptive graph representation of the brain network based on topological functional connectivity rather than distance-based connections. Further, non-contributory EEG channels are excluded by selecting only functional regions relevant to the corresponding intention. We empirically show that the proposed approach outperforms the state-of-the-art, with around 1% and 11% improvements over CNN-based and GNN-based models, on performing motor imagery predictions. Also, the task-adaptive channel selection demonstrates similar predictive performance with only 20% of raw EEG data, suggesting a possible shift in direction for future works other than simply scaling up the model.

A Spiking Neural Network with Adaptive Graph Convolution and LSTM for EEG-Based Brain-Computer Interfaces.

Electroencephalography (EEG) signals classification is essential for the brain-computer interface (BCI). Recently, energy-efficient spiking neural networks (SNNs) have shown great potential in EEG analysis due to their ability to capture the complex dynamic properties of biological neurons while also processing stimulus information through precisely timed spike trains. However, most existing methods do not effectively mine the specific spatial topology of EEG channels and temporal dependencies of the encoded EEG spikes. Moreover, most are designed for specific BCI tasks and lack some generality. Hence, this study presents a novel SNN model with the customized spike-based adaptive graph convolution and long short-term memory (LSTM), termed SGLNet, for EEG-based BCIs. Specifically, we first adopt a learnable spike encoder to convert the raw EEG signals into spike trains. Then, we tailor the concepts of the multi-head adaptive graph convolution to SNN so that it can make good use of the intrinsic spatial topology information among distinct EEG channels. Finally, we design the spike-based LSTM units to further capture the temporal dependencies of the spikes. We evaluate our proposed model on two publicly available datasets from two representative fields of BCI, notably emotion recognition, and motor imagery decoding. The empirical evaluations demonstrate that SGLNet consistently outperforms existing state-of-the-art EEG classification algorithms. This work provides a new perspective for exploring high-performance SNNs for future BCIs with rich spatiotemporal dynamics.

User Identity Protection in EEG-based Brain-Computer Interfaces: Supplementary Material.

A brain-computer interface (BCI) establishes a direct communication pathway between the brain and an external device. Electroencephalogram (EEG) is the most popular input signal in BCIs, due to its convenience and low cost. Most research on EEG-based BCIs focuses on the accurate decoding of EEG signals; however, EEG signals also contain rich private information, e.g., user identity, emotion, and so on, which should be protected. This paper first exposes a serious privacy problem in EEG-based BCIs, i.e., the user identity in EEG data can be easily learned so that different sessions of EEG data from the same user can be associated together to more reliably mine private information. To address this issue, we further propose two approaches to convert the original EEG data into identity-unlearnable EEG data, i.e., removing the user identity information while maintaining the good performance on the primary BCI task. Experiments on seven EEG datasets from five different BCI paradigms showed that on average the generated identity-unlearnable EEG data can reduce the user identification accuracy from 70.01% to at most 21.36%, greatly facilitating user privacy protection in EEG-based BCIs.

Multi-Kernel Temporal and Spatial Convolution for EEG-Based Emotion Classification.

Deep learning using an end-to-end convolutional neural network (ConvNet) has been applied to several electroencephalography (EEG)-based brain-computer interface tasks to extract feature maps and classify the target output. However, the EEG analysis remains challenging since it requires consideration of various architectural design components that influence the representational ability of extracted features. This study proposes an EEG-based emotion classification model called the multi-kernel temporal and spatial convolution network (MultiT-S ConvNet). The multi-scale kernel is used in the model to learn various time resolutions, and separable convolutions are applied to find related spatial patterns. In addition, we enhanced both the temporal and spatial filters with a lightweight gating mechanism. To validate the performance and classification accuracy of MultiT-S ConvNet, we conduct subject-dependent and subject-independent experiments on EEG-based emotion datasets: DEAP and SEED. Compared with existing methods, MultiT-S ConvNet outperforms with higher accuracy results and a few trainable parameters. Moreover, the proposed multi-scale module in temporal filtering enables extracting a wide range of EEG representations, covering short- to long-wavelength components. This module could be further implemented in any model of EEG-based convolution networks, and its ability potentially improves the model's learning capacity.