Hybrid Brain-computer Interface System Research Articles

Design of an fNIRS-EEG hybrid terminal for wearable BCI systems.

The importance of brain-computer interfaces (BCI) is increasing, and various methods have been developed. Among the developed BCI methods, functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) are favored due to their non-invasive feature and compact device sizes. EEG monitors the electrical potentials generated by the activation of neurons, and fNIRS monitors the blood flow also generated by neurons, resulting in signals with different properties between the two methods. As the two BCI methods greatly differ in the characteristics of the acquired neural activity signals, for cases of estimating the intention or thought of a subject by BCI, it has been proven that further accurate information may be extracted by utilizing both methods simultaneously. Both systems are powered by electricity, and as EEG systems are greatly sensitive to electrical noises, application of two separate fNIRS and EEG systems together may result in electrical interference as the systems are required to be in contact with the skin and stray currents from the fNIRS system may flow along the surface of the skin into the EEG system. This research proposes a wearable fNIRS-EEG hybrid BCI system, where a single terminal is capable of operating both as a continuous wave fNIRS emitter and as a detector, and also as an EEG electrode. The system has been designed such that the fNIRS and EEG components are electrically separated to avoid electrical interference between each other. It is expected that by utilizing the developed fNIRS-EEG hybrid terminals, the development of BCI analysis may be further accelerated in various fields.

Application of hybrid SSVEP + P300 brain computer interface to control avatar movement in mobile virtual reality gaming environment

IntroductionThis research evaluated the feasibility of a hybrid SSVEP + P300 brain computer interface (BCI) for controlling the movement of an avatar in a virtual reality (VR) gaming environment (VR + BCI). Existing VR + BCI gaming environments have limitations, such as visual fatigue, a lower communication rate, minimum accuracy, and poor system comfort. Hence, there is a need for an optimized hybrid BCI system that can simultaneously evoke the strongest P300 and SSVEP potentials in the cortex. MethodsA BCI headset was coupled with a VR headset to generate a VR + BCI environment. The author developed a VR game in which the avatar’s movement is controlled using the user's cortical responses with the help of a BCI headset. Specifically designed visual stimuli were used in the proposed system to elicit the strongest possible responses from the user's brain. The proposed system also includes an auditory feedback mechanism to facilitate precise avatar movement. Results and conclusionsConventional P300 BCI and SSVEP BCI were also used to control the movements of the avatar, and their performance metrics were compared to those of the proposed system. The results demonstrated that the hybrid SSVEP + P300 BCI system was superior to the other systems for controlling avatar movement.

A Hybrid Brain–Computer Interface Combining P300 Potentials and Emotion Patterns for Detecting Awareness in Patients With Disorders of Consciousness

In this study, a hybrid brain-computer interface (BCI) system combining P300 potential and emotion patterns was proposed to improve the performance of awareness detection. Two video clips were flashed randomly to evoke the P300 potential, while a laughing or crying video clip was used to induce the corresponding emotion pattern. The subjects were asked to concentrate on the laughing or crying video clip cued by the instruction and to count the flashes of the corresponding video clip. Two layers of classification were developed. In the first layer, P300 detection and emotion recognition were performed separately using two support vector machine (SVM) classifiers. Specifically, the activation, spatial and connection patterns were fused in emotion recognition. In the second layer, the SVM scores of P300 detection and emotion recognition were fed into another SVM classifier to determine which video clip the subjects responded to. Six healthy subjects and eight patients with disorders of consciousness (DOC) were involved in the command-following experiment. The results showed that the accuracy of the hybrid BCI system was better than those of the single-modality systems. Furthermore, three patients were able to perform tasks (66%-72%) using our hybrid BCI, which indicated their residual awareness and emotion-related abilities.

Hybrid Brain–Computer Interface System Using Auditory Stimulation and Speech Imagination Paradigms for Assistive Technology

In the case of paralysis with visual and tactile impairments, brain–computer interfaces (BCIs) based on auditory and mental imagery paradigms are alternative methods for controlling external devices. This study demonstrates the use of a hybrid BCI via auditory stimulation and speech imagination for assistive technology. The proposed auditory BCI using Thai vowel and numeral stimulus patterns as well as multi-loudspeaker position settings for multi-command BCI are investigated. To avoid auditory stimulation during resting periods, a speech imagery method is used to enable an audio stimulator. We observe the classification efficiency of speech imagery and auditory BCIs from selected electroencephalogram channels using the proposed algorithms. We examine the efficiency of using the proposed BCI in the presence of background noise (speech). One command is created using the proposed speech-imagination paradigm. Four commands are created using the proposed auditory stimulation paradigm. We design an experiment to verify the proposed BCI paradigm and classification algorithms for real-time processing. The results show that the average classification accuracy of the proposed auditory BCI using numeral stimuli and scatter patterns without speech noise is 72.2% to 83.3%, respectively. The efficiency under background noise is approximately two times lower than that without background noise.

A Novel Hybrid Brain-Computer Interface Combining the Illusion-Induced VEP and SSVEP.

Traditional single-modality brain-computer interface (BCI) systems are limited by their reliance on a single characteristic of brain signals. To address this issue, incorporating multiple features from EEG signals can provide robust information to enhance BCI performance. In this study, we designed and implemented a novel hybrid paradigm that combined illusion-induced visual evoked potential (IVEP) and steady-state visual evoked potential (SSVEP) with the aim of leveraging their features simultaneously to improve system efficiency. The proposed paradigm was validated through two experimental studies, which encompassed feature analysis of IVEP with a static paradigm, and performance evaluation of hybrid paradigm in comparison with the conventional SSVEP paradigm. The characteristic analysis yielded significant differences in response waveforms among different motion illusions. The performance evaluation of the hybrid BCI demonstrates the advantage of integrating illusory stimuli into the SSVEP paradigm. This integration effectively enhanced the spatio-temporal features of EEG signals, resulting in higher classification accuracy and information transfer rate (ITR) within a short time window when compared to traditional SSVEP-BCI in four-command task. Furthermore, the questionnaire results of subjective estimation revealed that proposed hybrid BCI offers less eye fatigue, and potentially higher levels of concentration, physical condition, and mental condition for users. This work first introduced the IVEP signals in hybrid BCI system that could enhance performance efficiently, which is promising to fulfill the requirements for efficiency in practical BCI control systems.

뇌-컴퓨터 인터랙션 오탐 저감을 위한 뇌파-턱 근전도 통합 신호 제어 프로토콜 개발

Objective: The purpose of this study was to investigate whether electromyogram (EMG) induced by jaw-clenching can be accurately observed in steady-state visually evoked potential (SSVEP) signals at occipital lobes as well as temporal lobes. We also investigated a feasibility that the fused SSVEP data with the jaw-clenching EMG can be classified as a function of task types characterized by SSVEPs with and without the short and long clenching noise. Based on this hypothesis, we proposed a novel command protocol to effectively and efficiently reduce false alarms caused by using SSVEP brain-computer interface (BCI).<BR><BR>Background: Much attention has been paid to various brain-computer interfaces for neural rehabilitation and alternative communication channels for disabled people with severe neurological disorders. However, despite the interests in BCI, false alarms caused by interference effects between command flickers with different frequencies or internal and external noises have been still problematic and have yet to be investigated. Even though protocols to correct the false alarms based on error-related potential (ErrP) or eye blinks have been proposed, it has still limitation due to errors inherent in involuntary reactions of humans. Thus, it is necessary to develop a protocol to address the issue and propose a method insusceptible to the involuntary reactions.<BR><BR>Method: Nine undergraduate students (3 female) voluntarily participated in the experiment. They were asked to divide their attention between the two squares randomly oscillating with 8.57 and 10Hz and perform three types of tasks characterized by SSVEP only, short clenching, and long clenching conditions. They carried out the three tasks with a semi-counterbalanced and random order and rest interval between the tasks was set to 10 minutes to minimize order and carryover effects. It took about 12 minutes in completing the experimental tasks for each participant. We used five machine learning algorithms of binary classification to compare the performance in classifying the brainwave signals characterized by the three tasks.<BR><BR>Results: As a result of the classification comparison, random forest showed the highest classification performance of more than 0.85 (AUC) as follows: 0.88 (short clenching vs. SSVEP), 0.87 (long clenching vs. SSVEP), 0.85 (long clenching vs. short clenching condition) in the test data set. Notably, the findings implicated a clear classification in the long cand short clenching condition even at the frequency band of more than 20Hz. For the participants whose highest weight factor was shown at the occipital lobe, it can be possible to reduce false alarms in the SSVEP BCI with only single electrode at the occipital lobe.<BR><BR>Conclusion: The findings found in this study implicate that the proposed protocol can be used to effectively and efficiently correct false alarms in event-related potential (ERP), event-related desynchronization/synchronization (ERD/ERS) BCI as well as SSVEP BCI. On top of that, classifying the subtle changes in the jaw clenching types enables BCI developers to create additional commands to modulate their BCI system. The results can be greatly improved if weight factors in the binary classification are modulated as a function of individual characteristics.<BR><BR>Application: The findings obtained in this study can be utilized in developing asynchronous BCI very robust to false alarms and expected to provide a future direction into hybrid BCI system.

Usability of a Hybrid System Combining P300-Based Brain-Computer Interface and Commercial Assistive Technologies to Enhance Communication in People With Multiple Sclerosis

Brain-computer interface (BCI) can provide people with motor disabilities with an alternative channel to access assistive technology (AT) software for communication and environmental interaction. Multiple sclerosis (MS) is a chronic disease of the central nervous system that mostly starts in young adulthood and often leads to a long-term disability, possibly exacerbated by the presence of fatigue. Patients with MS have been rarely considered as potential BCI end-users. In this pilot study, we evaluated the usability of a hybrid BCI (h-BCI) system that enables both a P300-based BCI and conventional input devices (i.e., muscular dependent) to access mainstream applications through the widely used AT software for communication “Grid 3.” The evaluation was performed according to the principles of the user-centered design (UCD) with the aim of providing patients with MS with an alternative control channel (i.e., BCI), potentially less sensitive to fatigue. A total of 13 patients with MS were enrolled. In session I, participants were presented with a widely validated P300-based BCI (P3-speller); in session II, they had to operate Grid 3 to access three mainstream applications with (1) an AT conventional input device and (2) the h-BCI. Eight patients completed the protocol. Five out of eight patients with MS were successfully able to access the Grid 3 via the BCI, with a mean online accuracy of 83.3% (± 14.6). Effectiveness (online accuracy), satisfaction, and workload were comparable between the conventional AT inputs and the BCI channel in controlling the Grid 3. As expected, the efficiency (time for correct selection) resulted to be significantly lower for the BCI with respect to the AT conventional channels (Z = 0.2, p < 0.05). Although cautious due to the limited sample size, these preliminary findings indicated that the BCI control channel did not have a detrimental effect with respect to conventional AT channels on the ability to operate an AT software (Grid 3). Therefore, we inferred that the usability of the two access modalities was comparable. The integration of BCI with commercial AT input devices to access a widely used AT software represents an important step toward the introduction of BCIs into the AT centers’ daily practice.

Toward a Brain-Computer Interface- and Internet of Things-Based Smart Ward Collaborative System Using Hybrid Signals.

This study proposes a brain-computer interface (BCI)- and Internet of Things (IoT)-based smart ward collaborative system using hybrid signals. The system is divided into hybrid asynchronous electroencephalography (EEG)-, electrooculography (EOG)- and gyro-based BCI control system and an IoT monitoring and management system. The hybrid BCI control system proposes a GUI paradigm with cursor movement. The user uses the gyro to control the cursor area selection and uses blink-related EOG to control the cursor click. Meanwhile, the attention-related EEG signals are classified based on a support-vector machine (SVM) to make the final judgment. The judgment of the cursor area and the judgment of the attention state are reduced, thereby reducing the false operation rate in the hybrid BCI system. The accuracy in the hybrid BCI control system was 96.65 ± 1.44%, and the false operation rate and command response time were 0.89 ± 0.42 events/min and 2.65 ± 0.48 s, respectively. These results show the application potential of the hybrid BCI control system in daily tasks. In addition, we develop an architecture to connect intelligent things in a smart ward based on narrowband Internet of Things (NB-IoT) technology. The results demonstrate that our system provides superior communication transmission quality.

A comprehensive benchmark dataset for SSVEP-based hybrid BCI

The use of hybrid brain–computer interface (HBCI) systems have received much attention in recent years. However, there are few published datasets for HBCI systems. This paper provides a benchmark dataset for these systems. The provided dataset consist of data corresponding to three speller systems, including SSVEP-based BCI system and HBCI systems based on SSVEP-EMG and SSVEP-EOG. These spellers have been implemented using electroencephalogram (EEG) and other biosignals, including electromyogram (EMG) and electrooculogram (EOG). The virtual keyboard of spellers was composed of 9 visual flickers with stimulation frequencies ranged from 5.58 Hz to 11.11 Hz. The stimuli presentation was controlled by the psychophysics toolbox of Matlab. We conducted three experiments, using three speller systems under the similar situation. Ten subjects participated in a copy-spelling task experiment. For each system, the data included ten sessions corresponding to 10 times, which the subject spells the determined phrase. These HBCI systems were validated using NASA-TLX workload index and performance evaluation parameters, including the accuracy and the information transfer rate. Results showed that combining two control signals only when signals are derived from one organ of the body, might increase the workload. These datasets can be used as a comprehensive dataset to evaluate the performance of different HBCI systems. They are freely available from Http://maleki.semnan.ac.ir/.

Novel Hybrid Brain-Computer Interface for Virtual Reality Applications Using Steady-State Visual-Evoked Potential-Based Brain-Computer Interface and Electrooculogram-Based Eye Tracking for Increased Information Transfer Rate.

Brain–computer interfaces (BCIs) based on electroencephalogram (EEG) have recently attracted increasing attention in virtual reality (VR) applications as a promising tool for controlling virtual objects or generating commands in a “hands-free” manner. Video-oculography (VOG) has been frequently used as a tool to improve BCI performance by identifying the gaze location on the screen, however, current VOG devices are generally too expensive to be embedded in practical low-cost VR head-mounted display (HMD) systems. In this study, we proposed a novel calibration-free hybrid BCI system combining steady-state visual-evoked potential (SSVEP)-based BCI and electrooculogram (EOG)-based eye tracking to increase the information transfer rate (ITR) of a nine-target SSVEP-based BCI in VR environment. Experiments were repeated on three different frequency configurations of pattern-reversal checkerboard stimuli arranged in a 3 × 3 matrix. When a user was staring at one of the nine visual stimuli, the column containing the target stimulus was first identified based on the user’s horizontal eye movement direction (left, middle, or right) classified using horizontal EOG recorded from a pair of electrodes that can be readily incorporated with any existing VR-HMD systems. Note that the EOG can be recorded using the same amplifier for recording SSVEP, unlike the VOG system. Then, the target visual stimulus was identified among the three visual stimuli vertically arranged in the selected column using the extension of multivariate synchronization index (EMSI) algorithm, one of the widely used SSVEP detection algorithms. In our experiments with 20 participants wearing a commercial VR-HMD system, it was shown that both the accuracy and ITR of the proposed hybrid BCI were significantly increased compared to those of the traditional SSVEP-based BCI in VR environment.

A Hybrid Brain-Computer Interface Based on Visual Evoked Potential and Pupillary Response.

Brain-computer interface (BCI) based on steady-state visual evoked potential (SSVEP) has been widely studied due to the high information transfer rate (ITR), little user training, and wide subject applicability. However, there are also disadvantages such as visual discomfort and “BCI illiteracy.” To address these problems, this study proposes to use low-frequency stimulations (12 classes, 0.8–2.12 Hz with an interval of 0.12 Hz), which can simultaneously elicit visual evoked potential (VEP) and pupillary response (PR) to construct a hybrid BCI (h-BCI) system. Classification accuracy was calculated using supervised and unsupervised methods, respectively, and the hybrid accuracy was obtained using a decision fusion method to combine the information of VEP and PR. Online experimental results from 10 subjects showed that the averaged accuracy was 94.90 ± 2.34% (data length 1.5 s) for the supervised method and 91.88 ± 3.68% (data length 4 s) for the unsupervised method, which correspond to the ITR of 64.35 ± 3.07 bits/min (bpm) and 33.19 ± 2.38 bpm, respectively. Notably, the hybrid method achieved higher accuracy and ITR than that of VEP and PR for most subjects, especially for the short data length. Together with the subjects’ feedback on user experience, these results indicate that the proposed h-BCI with the low-frequency stimulation paradigm is more comfortable and favorable than the traditional SSVEP-BCI paradigm using the alpha frequency range.

An autonomous hybrid brain-computer interface system combined with eye-tracking in virtual environment

BackgroundBrain-computer interface (BCI) has become an effective human-machine interactive way. However, the performance of the traditional BCI system needs to be further improved, such as flexibility, robustness, and accuracy. We aim to develop an autonomous hybrid BCI system combined with eye-tracking for the control tasks in the virtual environment. New methodThis work developed an autonomous control strategy and proposed an effective fusion method for electroencephalogram (EEG) and eye tracking. For the autonomous control, the sliding window method was adopted to analyze the user's eye-gaze data. When the variance of eye-gaze data was less than the threshold, target recognition was triggered. EEG and eye-gaze data were synchronously collected and fused for classification. In addition, a fusion method based on particle swarm optimization (PSO) was proposed, which can find the best fusion weights to adapt to the differences of single modalities. ResultsEEG data and eye-gaze data of 15 subjects in steady-state visual evoked potentials (SSVEP) tasks were collected to evaluate the effectiveness of the hybrid BCI system. The results showed that the PSO fusion method performed best in all fusion methods. And the proposed hybrid BCI system obtained higher accuracy and information transfer rate (ITR) than the single-modality. Comparison with existing methodsThe PSO fusion method was compared with average weighting fusion, prior weighting fusion, support vector machine, decision tree, random forest, and extreme random tree. ConclusionThe proposed methods of autonomous control and dual-modal fusion can improve the flexibility, robustness and classification performance of the hybrid BCI system.

Corticomuscular and Intermuscular Coupling in Simple Hand Movements to Enable a Hybrid Brain–Computer Interface

Hybrid Brain-Computer Interfaces (BCIs) for upper limb rehabilitation after stroke should enable the reinforcement of "more normal" brain and muscular activity. Here, we propose the combination of corticomuscular coherence (CMC) and intermuscular coherence (IMC) as control features for a novel hybrid BCI for rehabilitation purposes. Multiple electroencephalographic (EEG) signals and surface electromyography (EMG) from 5 muscles per side were collected in 20 healthy participants performing finger extension (Ext) and grasping (Grasp) with both dominant and non-dominant hand. Grand average of CMC and IMC patterns showed a bilateral sensorimotor area as well as multiple muscles involvement. CMC and IMC values were used as features to classify each task versus rest and Ext versus Grasp. We demonstrated that a combination of CMC and IMC features allows for classification of both movements versus rest with better performance (Area Under the receiver operating characteristic Curve, AUC) for the Ext movement (0.97) with respect to Grasp (0.88). Classification of Ext versus Grasp also showed high performances (0.99). All in all, these preliminary findings indicate that the combination of CMC and IMC could provide for a comprehensive framework for simple hand movements to eventually be employed in a hybrid BCI system for post-stroke rehabilitation.

A hybrid environment control system combining EMG and SSVEP signal based on brain-computer interface technology

The patients who are impaired with neurodegenerative disorders cannot command their muscles through the neural pathways. These patients are given an alternative from their neural path through Brain-Computer Interface (BCI) systems, which are the explicit use of brain impulses without any need for a computer's vocal muscle. Nowadays, the steady-state visual evoked potential (SSVEP) modality offers a robust communication pathway to introduce a non-invasive BCI. There are some crucial constituents, including window length of SSVEP response, the number of electrodes in the acquisition device and system accuracy, which are the critical performance components in any BCI system based on SSVEP signal. In this study, a real-time hybrid BCI system consists of SSVEP and EMG has been proposed for the environmental control system. The feature in terms of the common spatial pattern (CSP) has been extracted from four classes of SSVEP response, and extracted feature has been classified using K-nearest neighbors (k-NN) based classification algorithm. The obtained classification accuracy of eight participants was 97.41%. Finally, a control mechanism that aims to apply for the environmental control system has also been developed. The proposed system can identify 18 commands (i.e., 16 control commands using SSVEP and two commands using EMG). This result represents very encouraging performance to handle real-time SSVEP based BCI system consists of a small number of electrodes. The proposed framework can offer a convenient user interface and a reliable control method for realistic BCI technology.

Exploration of illusory visual motion stimuli: An EEG-based brain-computer interface for practical assistive communication systems

This paper presents an illusory visual motion stimulus-based brain-computer interface (BCI). We aim to use the proposed system to enhance the motor imagery (MI) modality. Since motor imagery requires a long time for training, a stimulation method with external stimuli through the sensory system is an alternative method for increasing efficiency. The research is divided into two parts. First, we observed the visual motion illusion pattern based on brain topographic maps for the novel BCI modality. Second, we implemented the illusory visual motion stimulus-based BCI system. Arrow and moving-arrow patterns were used to modulate alpha rhythms at the visual and motor cortex. The arrow pattern had an average classification accuracy of approximately 78.5%. Additionally, illusory visual motion stimulus-based BCI systems are proposed using the proposed feature extraction and decision-making algorithm. This proposed BCI system can control the cursor moving in the left or right direction with the designed algorithm to create five commands for assistive communication. Ten volunteers participated in the experiment, and a brain-computer interface system with motor imagery and an illusory visual motion stimulus were used to compare efficiencies. The results showed that the proposed method achieved approximately 4% higher accuracy than motor imagery. The accuracy of the proposed illusory visual motion stimulus and algorithm was approximately 80.3%. Therefore, an illusory visual motion stimulus hybrid BCI system can be incorporated into the MI-based BCI system for beginner motor imagery. Based on the results, the proposed assistive communication system can be used to enhance communication in people with severe disabilities.

SSVEP-assisted RSVP brain–computer interface paradigm for multi-target classification

Brain–computer Interface (BCI) is actively involved in optimizing the communication medium between the human brain and external devices.Objective. Rapid serial visual presentation (RSVP) is a robust and highly efficient BCI technique in recognizing target objects but suffers from limited target selections. Hybrid BCI systems that combine steady-state visual evoked potential (SSVEP) and RSVP can mitigate this limitation and allow users to operate on multiple targets.Approach. This study proposes a novel hybrid SSVEP-RSVP BCI to improve the performance of classifying the target/non-target objects in a multi-target scenario. In this paradigm, SSVEP stimulation helps in identifying the user’s focus location and RSVP stimuli that elicit event-related potentials differentiate target and non-target objects. Main results. The proposed model achieved an offline accuracy of 81.59% by using 12 electroencephalography (EEG) channels and an online (real-time) accuracy of 78.10% when only four EEG channels are considered. Further, the biomarkers of physiological states are analyzed to assess the cognitive states (mental fatigue and user attention) of the participants based on resting theta and alpha band powers. The results indicate an inverse relationship between the BCI performance and the resting EEG power, validating that the subjects’ performance is affected by physiological states for long-term use of the BCI. Significance. Our findings demonstrate that the combination of SSVEP and RSVP stimuli improves the BCI performance and further enhances the possibility of performing multiple user command tasks, which are inevitable in real-world applications. Additionally, the cognitive state biomarkers discussed imply the need for an efficient and attractive experimental paradigm that reduces the physiological state disparities and provide enhanced BCI performance.

Classification of working memory loads using hybrid EEG and fNIRS in machine learning paradigm

Single modality brain–computer interface (BCI) systems often mislabel the electroencephalography (EEG) signs as a command, even though the participant is not executing some task. In this Letter, the classification of different working memory load levels is presented using a hybrid BCI system. N-back cognitive tasks such as 0-back, 2-back, and 3-back are used to create working memory load on participants while recording EEG and functional near-infrared spectroscopy (fNIRS) signals simultaneously. A combination of statistically significant features obtained from EEG and fNIRS corresponding to frontal region channels are used to classify different N-back commands. Kernel-based support vector machine (SVM) classifiers are employed with and without cross-validation schemes. Classification accuracy of 100% is achieved for binary classification of 0-back against 2-back and 0-back against 3-back using linear SVM, quadratic SVM, and cubic SVM under holdout data division protocol.

A Hybrid BCI Using Singular Value Decomposition Values of the Fast Walsh–Hadamard Transform Coefficients

One of the main goals of a brain computer interface (BCI) is to enable a communication channel between the brain and electronic devices by converting neural activity into control commands either for devices or applications. Because of the excellent temporal resolution, low set-up cost, and non-invasive nature, BCI systems generally use electroencephalography (EEG) for an input signal. However, EEG suffers from poor spatial resolution, and it is contaminated by various external and internal artifacts such as environmental magnetic noises and body movements. These limitations directly affect the performance of the EEG-based BCI system, and it might not work at the desired level. On the other hand, near-infrared spectroscopy (NIRS) has an advantage of relative robustness against body movements and electrical artifacts. Additionally, it is also a promising neural signal recording method which provides good spatial resolution. In this study, we particularly focused on compensating the limitations of EEG-based BCI system by adding simultaneous NIRS modality features. In order to show the effectiveness of our method, we used an open-access dataset, which was recorded from 29 subjects with simultaneous EEG-NIRS system during the imagination of opening and closing either a left-or right-hand. The features were extracted by calculating the singular value decomposition values of the Fast Walsh-Hadamard Transform coefficients. Afterward, the k-nearest neighbor algorithm was performed to classify the features. The performance of the proposed method was evaluated in terms of classification accuracy and kappa value metrics. The achieved results showed that combining a hybrid BCI system with EEG-NIRS modalities can enhance the performance of a BCI by 6.75% compared to the single-modality solution of EEG.

A Bipolar-Channel Hybrid Brain-Computer Interface System for Home Automation Control Utilizing Steady-State Visually Evoked Potential and Eye-Blink Signals.

The goal of this study was to develop and validate a hybrid brain-computer interface (BCI) system for home automation control. Over the past decade, BCIs represent a promising possibility in the field of medical (e.g., neuronal rehabilitation), educational, mind reading, and remote communication. However, BCI is still difficult to use in daily life because of the challenges of the unfriendly head device, lower classification accuracy, high cost, and complex operation. In this study, we propose a hybrid BCI system for home automation control with two brain signals acquiring electrodes and simple tasks, which only requires the subject to focus on the stimulus and eye blink. The stimulus is utilized to select commands by generating steady-state visually evoked potential (SSVEP). The single eye blinks (i.e., confirm the selection) and double eye blinks (i.e., deny and re-selection) are employed to calibrate the SSVEP command. Besides that, the short-time Fourier transform and convolution neural network algorithms are utilized for feature extraction and classification, respectively. The results show that the proposed system could provide 38 control commands with a 2 s time window and a good accuracy (i.e., 96.92%) using one bipolar electroencephalogram (EEG) channel. This work presents a novel BCI approach for the home automation application based on SSVEP and eye blink signals, which could be useful for the disabled. In addition, the provided strategy of this study—a friendly channel configuration (i.e., one bipolar EEG channel), high accuracy, multiple commands, and short response time—might also offer a reference for the other BCI controlled applications.

A Synchronized Hybrid Brain-Computer Interface System for Simultaneous Detection and Classification of Fusion EEG Signals

Brain-computer interface (BCI) technology represents a fast-growing field of research and applications for disabled and healthy people, which is a direct communication pathway to translate the neural information into an active command. Owing to the complicated headset structure, low accuracies, extended training periods, and nonstationary noises, BCI still has many challenges that should be dealt with for further facilitation of BCI technology use in daily life. In this study, a simplified synchronized hybrid BCI system is proposed for multiple command control by the electroencephalograph (EEG) signals in the motor cortex. This system can detect the single motor imagery (MI) task, single steady-state visually evoked potential (SSVEP) task, and hybrid MI + SSVEP tasks simultaneously (total ten mental tasks) via 2 EEG channels with high accuracy. The fast independent component analysis algorithm is employed to hybrid signals for obtaining clear EEG signals resulting from denoising. Feature extraction is performed by the wavelet transform, which is extracted by the features in the frequency and time domains. Furthermore, a four-layer convolutional neural network (CNN) is used as a classifier to distinguish different mental tasks. Finally, the hybrid MI + SSVEP system with a simple structure achieves a high accuracy of 95.56%. Additionally, the single MI-based and the SSVEP-based BCI system obtain the classification accuracy of 90.16% and 93.21%, respectively. Experimental results indicate that the synchronized hybrid BCI system could achieve multiple command control with a simple structure. In comparison with the single MI-based and the SSVEP-based BCI system, the hybrid MI + SSVEP BCI system shows a stable performance and higher efficiency. The proposed investigation provides a new method for the multiple command control by a hybrid BCI system. Also, the proposed BCI system offers the possibility of friendly utilization for disabled people because of its reliability, ease of use, and simplified headset structure.