Real-time Brain-computer Interface System Research Articles

Model and Data Dual-Driven Double-Point Observation Network for Ultra-Short MI EEG Classification.

Although deep networks have succeeded in various signal classification tasks, the time sequence samples used to train the deep models are usually required to reach a certain length. Especially, in brain computer interface (BCI) research, around 3.5s-long motor imagery (MI) Electroencephalography (EEG) samples are needed to obtain satisfactory classification performance. This time-span requirement of the training samples makes real-time MI BCI systems impossible to implement based on deep networks, which restricts the related researches within laboratory and makes practical application hard to accomplish. To address this issue, a double-point observation deep network (DoNet) is developed to classify ultra-short samples buried in noise. First, an analytical solution is developed theoretically to perform ultra-short signal classification based on double-point couples. Then, a signal-noise model is constructed to study the interference of noise on classification based on double-point couples. Based on which, an independent identical distribution condition is utilized to improve the classification accuracy in a data-driven manner. Combining the theoretical model and data-driven mechanism, DoNet can construct a steady data-distribution for the double-point couples of the samples with the same label. Therefore, the conditional probability of each double-point couple of a test sample can be obtained. With a voting strategy, the samples can be accurately classified by fusing these conditional probabilities. Meanwhile, the noise interference can be suppressed. DoNet has been evaluated on two public EEG datasets. Compared to most state-of-the-art methods, the 1s-long EEG signal classification accuracy has been improved by more than 3%.

A novel framework for classification of two-class motor imagery EEG signals using logistic regression classification algorithm.

Robotics and artificial intelligence have played a significant role in developing assistive technologies for people with motor disabilities. Brain-Computer Interface (BCI) is a communication system that allows humans to communicate with their environment by detecting and quantifying control signals produced from different modalities and translating them into voluntary commands for actuating an external device. For that purpose, classification the brain signals with a very high accuracy and minimization of the errors is of profound importance to the researchers. So in this study, a novel framework has been proposed to classify the binary-class electroencephalogram (EEG) data. The proposed framework is tested on BCI Competition IV dataset 1 and BCI Competition III dataset 4a. Artifact removal from EEG data is done through preprocessing, followed by feature extraction for recognizing discriminative information in the recorded brain signals. Signal preprocessing involves the application of independent component analysis (ICA) on raw EEG data, accompanied by the employment of common spatial pattern (CSP) and log-variance for extracting useful features. Six different classification algorithms, namely support vector machine, linear discriminant analysis, k-nearest neighbor, naïve Bayes, decision trees, and logistic regression, have been compared to classify the EEG data accurately. The proposed framework achieved the best classification accuracies with logistic regression classifier for both datasets. Average classification accuracy of 90.42% has been attained on BCI Competition IV dataset 1 for seven different subjects, while for BCI Competition III dataset 4a, an average accuracy of 95.42% has been attained on five subjects. This indicates that the model can be used in real time BCI systems and provide extra-ordinary results for 2-class Motor Imagery (MI) signals classification applications and with some modifications this framework can also be made compatible for multi-class classification in the future.

Finger pinching and imagination classification: A fusion of CNN architectures for IoMT-enabled BCI applications

A Brain–Computer Interface (BCI), integrated with the Internet of Medical Things (IoMT) and based on electroencephalogram (EEG) technology, allows users to control external devices by decoding brainwave patterns. Advanced deep learning-based BCIs, especially those utilizing sensorimotor rhythms (SMRs), have emerged as direct brain–device communication facilitators. SMRs involve users imagining limb motions to induce specific brain activity changes in the motor cortex. Despite progress, some users struggle with BCIs due to weak signals, individual variability, and limited task applicability. This study introduces an unsupervised EEG preprocessing pipeline for SMR-based BCIs. It evaluates an EEG dataset recorded during finger movements, employing two cleaning methods: an investigator-dependent pipeline and our proposed unsupervised method. Two distinct feature datasets are generated: one from cleaned EEG data processed into spectrogram images using supervised preprocessing, and another from data cleaned using our proposed unsupervised pipeline. The study extensively assesses five transfer learning convolutional neural network (TL-CNN) models for distinguishing Motor Imagery (MI) from finger movements (Mex) using the generated datasets. A novel probability fusion technique is developed to enhance TL-CNN classification in Mex versus MI finger-pinching actions. Comparative results show that the fusion-based method outperforms other state-of-the-art methods when applied to unsupervised EEG data. Specifically, our proposed approach achieves 97.9% accuracy, 93.4% precision, 95% recall, and an F1-score of 93.2%, demonstrating significant progress in distinguishing MI and Mex activities through the use of our unsupervised pre-processing pipeline and fusion-based CNN method. Our findings demonstrate the potential of our approach to serve as a benchmark for the global interdisciplinary research community and enable the development of future more effective and user-friendly real-time BCI systems.

Using occipital ⍺-bursts to modulate behavior in real-time.

Pre-stimulus endogenous neural activity can influence the processing of upcoming sensory input and subsequent behavioral reactions. Despite it is known that spontaneous oscillatory activity mostly appears in stochastic bursts, typical approaches based on trial averaging fail to capture this. We aimed at relating spontaneous oscillatory bursts in the alpha band (8-13Hz) to visual detection behavior, via an electroencephalography-based brain-computer interface (BCI) that allowed for burst-triggered stimulus presentation in real-time. According to alpha theories, we hypothesized that visual targets presented during alpha-bursts should lead to slower responses and higher miss rates, whereas targets presented in the absence of bursts (low alpha activity) should lead to faster responses and higher false alarm rates. Our findings support the role of bursts of alpha oscillations in visual perception and exemplify how real-time BCI systems can be used as a test bench for brain-behavioral theories.

Multivariate variational mode decomposition & phase space reconstruction based motor imagery EEG classification

Electroencephalogram signals have gained much attention in recent years as these signals help in decoding a person's intentions for performing various actions using the Brain Computer Interface system. However, it is challenging to accurately classify human intentions as the electroencephalogram signals are non-linear and non-stationary. This work proposes a novel methodology for feature extraction and classification of motor imagery electroencephalogram signals. The proposed methodology is based on Multivariate Variational Mode Decomposition and Phase Space Reconstruction. A number of statistical and non-linear features are extracted and classified using the Ensemble Support Vector Machine model. The efficiency of the methodology is evaluated on BCI competition-IV(2a) dataset. The evaluation results conclude that the proposed methodology is able to classify both binary and multi-class electroencephalogram data with an average classification accuracy of 96.76% and 84.22%, respectively. Hence, the proposed methodology can be utilized in designing a real-time Brain Computer interface system.

Detection of Movement Related Cortical Potentials from Single Trial EEG Signals

Movement-Related Cortical Potentials (MRCP) are signals that begin to appear approximately two seconds before the onset of voluntary movements and can be recorded with EEG. MRCP is an important sign that the movement will begin. Determining the movement intention before the action is extremely important information especially for real-time BCI systems. By using MRCP, Brain-Computer Interface (BCI) users' movement intention can be determined prior to the move and this sign can be used as a control signal. In this study, it was aimed to determine the movement and resting states with high accuracy with MRCP signals. Furthermore, the effects of filter cutoff frequencies, number of electrodes, and MRCP time interval window on the success of distinguishing movement/resting states in the preprocessing stage were investigated. For this purpose, Katz fractal dimension and nonlinear support vector machine methods were used in the feature extraction and classification stages, respectively. The proposed method was tested on the attempted hand and arm movements dataset containing EEG signals of 10 participants with spinal cord injury. Katz fractal dimension and support vector machines methods can determine movement and resting states with an average of 96.47% accuracy using MRCP signals. If the number of electrodes to be used in signal analysis was 3, 9 and 61, the obtained accuracy rates were determined as 83.71%, 90.67%, and 96.47%, respectively. The experimental results also showed that the filter cutoff frequencies used in the preprocessing had a significant effect on the accuracy.

Noninvasive neuroimaging and spatial filter transform enable ultra low delay motor imagery EEG decoding

Objective. The brain-computer interface (BCI) system based on sensorimotor rhythm can convert the human spirit into instructions for machine control, and it is a new human-computer interaction system with broad applications. However, the spatial resolution of scalp electroencephalogram (EEG) is limited due to the presence of volume conduction effects. Therefore, it is very meaningful to explore intracranial activities in a noninvasive way and improve the spatial resolution of EEG. Meanwhile, low-delay decoding is an essential factor for the development of a real-time BCI system. Approach. In this paper, EEG conduction is modeled by using public head anatomical templates, and cortical EEG is obtained using dynamic parameter statistical mapping. To solve the problem of a large amount of computation caused by the increase in the number of channels, the filter bank common spatial pattern method is used to obtain a spatial filter kernel, which reduces the computational cost of feature extraction to a linear level. And the feature classification and selection of important features are completed using a neural network containing band-spatial-time domain self-attention mechanisms. Main results. The results show that the method proposed in this paper achieves high accuracy for the four types of motor imagery EEG classification tasks, with fairly low latency and high physiological interpretability. Significance. The proposed decoding framework facilitates the realization of low-latency human-computer interaction systems.

On-Device Implementation for Deep-Learning-Based Cognitive Activity Prediction

Cognitive activity prediction (CAP) from electroencephalogram (EEG) signals is progressively utilized in the field of brain–computer interface (BCI) and mental health management. Various machine and deep learning methods have been proposed recently for CAP. However, since Internet-of-Things-based real-time BCI systems demand low latency, power, and portability, these methods need to be deployable on resource-constrained edge devices. Towards this aspect, we propose a real-time implementation of a lightweight 1-D convolutional neural network on an Arduino Due microcontroller for CAP from EEG signals. The performance evaluation on two public datasets and one real-time recorded dataset indicates that the proposed work achieves subject-independent prediction accuracies of 99.30%, 82.50%, and 99.02% in these datasets. Furthermore, the prediction of real-time recorded EEG signals is accurate for majority of the subjects. The proposed work outperforms the existing techniques and achieves low power consumption of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{0.63}\,\text{W}$</tex-math></inline-formula> in real-time on-device implementation with an average latency of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{455.12}\,\text{ms}$</tex-math></inline-formula> in model deployment, test output prediction, and activity-based transmission.

A Fast Approach to Removing Muscle Artifacts for EEG with Signal Serialization Based Ensemble Empirical Mode Decomposition.

An electroencephalogram (EEG) is an electrophysiological signal reflecting the functional state of the brain. As the control signal of the brain–computer interface (BCI), EEG may build a bridge between humans and computers to improve the life quality for patients with movement disorders. The collected EEG signals are extremely susceptible to the contamination of electromyography (EMG) artifacts, affecting their original characteristics. Therefore, EEG denoising is an essential preprocessing step in any BCI system. Previous studies have confirmed that the combination of ensemble empirical mode decomposition (EEMD) and canonical correlation analysis (CCA) can effectively suppress EMG artifacts. However, the time-consuming iterative process of EEMD may limit the application of the EEMD-CCA method in real-time monitoring of BCI. Compared with the existing EEMD, the recently proposed signal serialization based EEMD (sEEMD) is a good choice to provide effective signal analysis and fast mode decomposition. In this study, an EMG denoising method based on sEEMD and CCA is discussed. All of the analyses are carried out on semi-simulated data. The results show that, in terms of frequency and amplitude, the intrinsic mode functions (IMFs) decomposed by sEEMD are consistent with the IMFs obtained by EEMD. There is no significant difference in the ability to separate EMG artifacts from EEG signals between the sEEMD-CCA method and the EEMD-CCA method (p > 0.05). Even in the case of heavy contamination (signal-to-noise ratio is less than 2 dB), the relative root mean squared error is about 0.3, and the average correlation coefficient remains above 0.9. The running speed of the sEEMD-CCA method to remove EMG artifacts is significantly improved in comparison with that of EEMD-CCA method (p < 0.05). The running time of the sEEMD-CCA method for three lengths of semi-simulated data is shortened by more than 50%. This indicates that sEEMD-CCA is a promising tool for EMG artifact removal in real-time BCI systems.

A hybrid environment control system combining EMG and SSVEP signal based on brain-computer interface technology

The patients who are impaired with neurodegenerative disorders cannot command their muscles through the neural pathways. These patients are given an alternative from their neural path through Brain-Computer Interface (BCI) systems, which are the explicit use of brain impulses without any need for a computer's vocal muscle. Nowadays, the steady-state visual evoked potential (SSVEP) modality offers a robust communication pathway to introduce a non-invasive BCI. There are some crucial constituents, including window length of SSVEP response, the number of electrodes in the acquisition device and system accuracy, which are the critical performance components in any BCI system based on SSVEP signal. In this study, a real-time hybrid BCI system consists of SSVEP and EMG has been proposed for the environmental control system. The feature in terms of the common spatial pattern (CSP) has been extracted from four classes of SSVEP response, and extracted feature has been classified using K-nearest neighbors (k-NN) based classification algorithm. The obtained classification accuracy of eight participants was 97.41%. Finally, a control mechanism that aims to apply for the environmental control system has also been developed. The proposed system can identify 18 commands (i.e., 16 control commands using SSVEP and two commands using EMG). This result represents very encouraging performance to handle real-time SSVEP based BCI system consists of a small number of electrodes. The proposed framework can offer a convenient user interface and a reliable control method for realistic BCI technology.

Interface, interaction, and intelligence in generalized brain–computer interfaces

A brain-computer interface (BCI) establishes a direct communication channel between a brain and an external device. With recent advances in neurotechnology and artificial intelligence (AI), the brain signals in BCI communication have been advanced from sensation and perception to higher-level cognition activities. While the field of BCI has grown rapidly in the past decades, the core technologies and innovative ideas behind seemingly unrelated BCI systems have never been summarized from an evolutionary point of view. Here, we review various BCI paradigms and present an evolutionary model of generalized BCI technology which comprises three stages: interface, interaction, and intelligence (I3). We also highlight challenges, opportunities, and future perspectives in the development of new BCI technology.

SPECTRA: a tool for enhanced brain wave signal recognition

BackgroundBrain wave signal recognition has gained increased attention in neuro-rehabilitation applications. This has driven the development of brain–computer interface (BCI) systems. Brain wave signals are acquired using electroencephalography (EEG) sensors, processed and decoded to identify the category to which the signal belongs. Once the signal category is determined, it can be used to control external devices. However, the success of such a system essentially relies on significant feature extraction and classification algorithms. One of the commonly used feature extraction technique for BCI systems is common spatial pattern (CSP).ResultsThe performance of the proposed spatial-frequency-temporal feature extraction (SPECTRA) predictor is analysed using three public benchmark datasets. Our proposed predictor outperformed other competing methods achieving lowest average error rates of 8.55%, 17.90% and 20.26%, and highest average kappa coefficient values of 0.829, 0.643 and 0.595 for BCI Competition III dataset IVa, BCI Competition IV dataset I and BCI Competition IV dataset IIb, respectively.ConclusionsOur proposed SPECTRA predictor effectively finds features that are more separable and shows improvement in brain wave signal recognition that can be instrumental in developing improved real-time BCI systems that are computationally efficient.

A Novel Simplified Convolutional Neural Network Classification Algorithm of Motor Imagery EEG Signals Based on Deep Learning

Left and right hand motor imagery electroencephalogram (MI-EEG) signals are widely used in brain-computer interface (BCI) systems to identify a participant intent in controlling external devices. However, due to a series of reasons, including low signal-to-noise ratios, there are great challenges for efficient motor imagery classification. The recognition of left and right hand MI-EEG signals is vital for the application of BCI systems. Recently, the method of deep learning has been successfully applied in pattern recognition and other fields. However, there are few effective deep learning algorithms applied to BCI systems, particularly for MI based BCI. In this paper, we propose an algorithm that combines continuous wavelet transform (CWT) and a simplified convolutional neural network (SCNN) to improve the recognition rate of MI-EEG signals. Using the CWT, the MI-EEG signals are mapped to time-frequency image signals. Then the image signals are input into the SCNN to extract the features and classify them. Tested by the BCI Competition IV Dataset 2b, the experimental results show that the average classification accuracy of the nine subjects is 83.2%, and the mean kappa value is 0.651, which is 11.9% higher than that of the champion in the BCI Competition IV. Compared with other algorithms, the proposed CWT-SCNN algorithm has a better classification performance and a shorter training time. Therefore, this algorithm could enhance the classification performance of MI based BCI and be applied in real-time BCI systems for use by disabled people.

DWT and CNN based multi-class motor imagery electroencephalographic signal recognition

Objective. Brain computer interface (BCI) system allows humans to control external devices through motor imagery (MI) signals. However, many existing feature extraction algorithms cannot eliminate the influence of individual differences. This research proposed a new processing algorithm that can reduce the impact of individual differences on classification and improve the universality of the algorithm. Approach. To select the optimal frequency band, the energy in each sub-band was calculated by the discrete wavelet transform. Power spectral density and visual geometric group network based convolutional neural network were used for feature extraction and classification respectively. Main results. The test of the BCI Competition IV dataset IIa proved the superiority of the algorithm. In comparison with some commonly used methods, the proposed algorithm reduced classification calculation time while improving classification accuracy; the average classification accuracy rate reaches 96.21%, which is far exceeding the results obtained by the latest literature. Significance. The good classification performance of this research was rooted in the reduced number of parameters, the reduced consumption of computing resources, and the eliminated influence of individual differences. Therefore, the proposed algorithm can be applied to a real-time multi-class BCI system.

Identifying motor imagery activities in brain computer interfaces based on the intelligent selection of most informative timeframe

Removing the contribution of dispensable mental activities dispersed across the electroencephalogram (EEG) signal improves the interpretability and efficiency of the intended neuronal responses to control a brain computer interface (BCI). Performing motor imagery tasks causes proper formation of special patterns at a specific timeframe of the EEG signal. The accurate selection of this optimal informative timeframe has not yet been investigated. Previous studies have considered an identic portion of data for all individuals, while neglecting that the duration and delay takes for the motor imagery brain activities to be well reflected in EEG signals vary between individuals. We propose an intelligent hybrid genetic algorithm—support vector machine (SVM) method to select the most stimulated timeframe of interest. The method also selects the most distinctive subset of features (through a comprehensive fused set of temporal, spectral and wavelet inspected information) while simultaneously optimize the SVM classifier’s parameters. Evaluation results show that not only the most stimulated timeframe has a short duration but also occurs after a specific delay: that they are different between individuals. Using this optimal timeframe, the classification accuracy increased up to 92.14% for Graz 2003 and 89.00%, 84.81% and 85.00% for O3, S4 and X11 subjects of Graz 2005 database respectively. These results that were obtained despite the use of a small set of features confirm that this intelligent method can be effective in increasing the computational speed while decreasing the computational complexity which provides the potential capability of including in real time BCI systems.

SSVEP Transient Feature Extraction and Rapid Recognition Method Based on Bistable Stochastic Resonance.

Steady-state Visual Evoked Potential, SSVEP), as the most commonly used communication paradigm for non-implantable Brain-Computer Interface (BCI), boasts the advantages of unnecessity of training, noise immunity and periodicity. The traditional SSVEP extraction methods can effectively identify the target frequency contained in original EEG, however, the required data length usually lasts a few seconds. In this paper, bistable stochastic resonance (BSR) is applied to SSVEP extraction. BSR is very sensitive to amplitude mutation and frequency fluctuation of the input signal, making the output difference can be used for the detection of the target frequency. The processing results illustrate that the proposed method not only has a high recognition accuracy, but also effectively shortens the recognition time, thus improving the calculating speed. Therefore, SSVEP extraction based on BSR has a higher information transmission rate (ITR), which is more suitable for the real-time BCI system.

Design and Implementation of Real-time Brain-computer Interface System Based on LabVIEW

Brain-computer interface (BCI) is a newly developed man-machine interface method that is applied in the field of brain science, rehabilitation engineering, biomedical engineering and human-machine automatic control. Under the LabVIEW environment of virtual instrument graphical programming language, this paper improves the program of Active One, a physiological signal acquisition system. The off-line processing system of electroencephalogram(EEG) is improved to a real-time processing system and a real-time feedback BCI experiment system is established based on visual evoked potential (VEP). The cumulative average method combined with the FIR filter method was used to extract the VEP signal and form a closed-loop BCI. Real-time feedback of the BCI system was realized and the VEP signal was automatically identified. Using the proposed method of this paper to implement BCI can achieve a higher accuracy with certain practicality and feasibility.

Feature Selection for Motor Imagery EEG Classification Based on Firefly Algorithm and Learning Automata.

Motor Imagery (MI) electroencephalography (EEG) is widely studied for its non-invasiveness, easy availability, portability, and high temporal resolution. As for MI EEG signal processing, the high dimensions of features represent a research challenge. It is necessary to eliminate redundant features, which not only create an additional overhead of managing the space complexity, but also might include outliers, thereby reducing classification accuracy. The firefly algorithm (FA) can adaptively select the best subset of features, and improve classification accuracy. However, the FA is easily entrapped in a local optimum. To solve this problem, this paper proposes a method of combining the firefly algorithm and learning automata (LA) to optimize feature selection for motor imagery EEG. We employed a method of combining common spatial pattern (CSP) and local characteristic-scale decomposition (LCD) algorithms to obtain a high dimensional feature set, and classified it by using the spectral regression discriminant analysis (SRDA) classifier. Both the fourth brain–computer interface competition data and real-time data acquired in our designed experiments were used to verify the validation of the proposed method. Compared with genetic and adaptive weight particle swarm optimization algorithms, the experimental results show that our proposed method effectively eliminates redundant features, and improves the classification accuracy of MI EEG signals. In addition, a real-time brain–computer interface system was implemented to verify the feasibility of our proposed methods being applied in practical brain–computer interface systems.

Influence of dual-tasking with different levels of attention diversion on characteristics of the movement-related cortical potential

Dual tasking is defined as performing two tasks concurrently and has been shown to have a significant effect on attention directed to the performance of the main task. In this study, an attention diversion task with two different levels was administered while participants had to complete a cue-based motor task consisting of foot dorsiflexion. An auditory oddball task with two levels of complexity was implemented to divert the user’s attention. Electroencephalographic (EEG) recordings were made from nine single channels. Event-related potentials (ERPs) confirmed that the oddball task of counting a sequence of two tones decreased the auditory P300 amplitude more than the oddball task of counting one target tone among three different tones. Pre-movement features quantified from the movement-related cortical potential (MRCP) were changed significantly between single and dual-task conditions in motor and fronto-central channels. There was a significant delay in movement detection for the case of single tone counting in two motor channels only (237.1–247.4ms). For the task of sequence counting, motor cortex and frontal channels showed a significant delay in MRCP detection (232.1–250.5ms). This study investigated the effect of attention diversion in dual-task conditions by analysing both ERPs and MRCPs in single channels. The higher attention diversion lead to a significant reduction in specific MRCP features of the motor task. These results suggest that attention division in dual-tasking situations plays an important role in movement execution and detection. This has important implications in designing real-time brain-computer interface systems.

Phase-based SSVEPs for real-time control of a motorised bed.

Brain-computer interface (BCI) systems have emerged as an augmentative technology that can provide a promising solution for individuals with motor dysfunctions and for the elderly who are experiencing muscle weakness. Steady-state visually evoked potentials (SSVEPs) are widely adopted in BCI systems due to their high speed and accuracy when compared to other BCI paradigms. In this paper, we apply combined magnitude and phase features for class discrimination in a real-time SSVEP-based BCI platform. In the proposed real-time system users gain control of a motorised bed system with seven motion commands and an idle state. Experimental results amongst eight participants demonstrate that the proposed real-time BCI system can successfully discriminate between different SSVEP signals achieving high information transfer rates (ITR) of 82.73 bits/min. The attractive features of the proposed system include noninvasive recording, simple electrode configuration, excellent BCI response and minimal training requirements.