Brain-computer Interfaces Modalities Research Articles

POMDP-BCI: A Benchmark of (Re)Active BCI Using POMDP to Issue Commands.

Past research in Brain-Computer Interfaces (BCI) have presented different decoding algorithms for different modalities. Meanwhile, highly specific decision making processes have been developed for some of these modalities, while others lack such a component in their classic pipeline. The present work proposes a model based on Partially Observable Markov Decission Process (POMDP) that works as a high-level decision making framework for three different active/reactive BCI modalities. We tested our approach on three different BCI modalities using publicly available datasets. We compared the general POMDP model as a decision making process with state of the art methods for each BCI modality. Accuracy, false positive (FP) trials, no-action (NA) trials and average decision time are presented as metrics. Our results show how the presented POMDP models achieve comparable or better performance to the presented baseline methods, while being usable for the three proposed experiments without significant changes. Crucially, it offers the possibility of taking no-action (NA) when the decoding does not perform well. The present work implements a flexible POMDP model that acts as a sequential decision framework for BCI systems that lack such a component, and perform comparably to those that include it. We believe the proposed POMDP framework provides several interesting properties for future BCI developments, mainly the generalizability to any BCI modality and the possible integration of other physiological or brain data pipelines under a unified decision-making framework.

Brain-Computer Interfaces: Investigating the Transition from Visually Evoked to Purely Imagined Steady-State Potentials.

Brain-Computer Interfaces (BCIs) based on Steady State Visually Evoked Potentials (SSVEPs) have proven effective and provide significant accuracy and information-transfer rates. This family of strategies, however, requires external devices that provide the frequency stimuli required by the technique. This limits the scenarios in which they can be applied, especially when compared to other BCI approaches. In this work, we have investigated the possibility of obtaining frequency responses in the EEG output based on the pure visual imagination of SSVEP-eliciting stimuli. Our results show that not only that EEG signals present frequency-specific peaks related to the frequency the user is focusing on, but also that promising classification accuracy can be achieved, paving the way for a robust and reliable visual imagery BCI modality. Clinical relevance-Brain computer interfaces play a fundamental role in enhancing the quality of life of patients with severe motor impairments. Strategies based on purely imagined stimuli, like the one presented here, are particularly impacting, especially in the most severe cases.

Performance Variation of a Somatosensory BCI Based on Imagined Sensation: A Large Population Study.

A proportion of users cannot achieve adequate brain-computer interface (BCI) control. The diversity of BCI modalities provides a way to solve this emerging issue. Here, we investigate the accuracy of a somatosensory BCI based on sensory imagery (SI). During the SI tasks, subjects were instructed to imagine a tactile sensation and to maintain the attention on the corresponding hand, as if there was tactile stimulus on the skin of the wrist. The performance across 106 healthy subjects in left- and right-hand SI discrimination was 78.9±13.2%. In 70.7% of the subjects the performance was above 70%. The SI task induced a contralateral cortical activation, and high-density EEG source localization showed that the real tactile stimulation and imagined tactile stimulation shared similar cortical activations within the somatosensory cortex. The somatosensory BCI based on SI provides a new signal modality for independent BCI development. Moreover, a combination of SI and other BCI modalities, such as motor imagery, may provide new avenues for further improving BCI usage and applicability, especially in those subjects unable to attain adequate BCI control with conventional BCI modalities.

A Domain Adaptation Sparse Representation Classifier for Cross-Domain Electroencephalogram-Based Emotion Classification.

The brain-computer interface (BCI) interprets the physiological information of the human brain in the process of consciousness activity. It builds a direct information transmission channel between the brain and the outside world. As the most common non-invasive BCI modality, electroencephalogram (EEG) plays an important role in the emotion recognition of BCI; however, due to the individual variability and non-stationary of EEG signals, the construction of EEG-based emotion classifiers for different subjects, different sessions, and different devices is an important research direction. Domain adaptation utilizes data or knowledge from more than one domain and focuses on transferring knowledge from the source domain (SD) to the target domain (TD), in which the EEG data may be collected from different subjects, sessions, or devices. In this study, a new domain adaptation sparse representation classifier (DASRC) is proposed to address the cross-domain EEG-based emotion classification. To reduce the differences in domain distribution, the local information preserved criterion is exploited to project the samples from SD and TD into a shared subspace. A common domain-invariant dictionary is learned in the projection subspace so that an inherent connection can be built between SD and TD. In addition, both principal component analysis (PCA) and Fisher criteria are exploited to promote the recognition ability of the learned dictionary. Besides, an optimization method is proposed to alternatively update the subspace and dictionary learning. The comparison of CSFDDL shows the feasibility and competitive performance for cross-subject and cross-dataset EEG-based emotion classification problems.

Vector Phase Analysis Approach for Sleep Stage Classification: A Functional Near-Infrared Spectroscopy-Based Passive Brain-Computer Interface.

A passive brain–computer interface (BCI) based upon functional near-infrared spectroscopy (fNIRS) brain signals is used for earlier detection of human drowsiness during driving tasks. This BCI modality acquired hemodynamic signals of 13 healthy subjects from the right dorsolateral prefrontal cortex (DPFC) of the brain. Drowsiness activity is recorded using a continuous-wave fNIRS system and eight channels over the right DPFC. During the experiment, sleep-deprived subjects drove a vehicle in a driving simulator while their cerebral oxygen regulation (CORE) state was continuously measured. Vector phase analysis (VPA) was used as a classifier to detect drowsiness state along with sleep stage-based threshold criteria. Extensive training and testing with various feature sets and classifiers are done to justify the adaptation of threshold criteria for any subject without requiring recalibration. Three statistical features (mean oxyhemoglobin, signal peak, and the sum of peaks) along with six VPA features (trajectory slopes of VPA indices) were used. The average accuracies for the five classifiers are 90.9% for discriminant analysis, 92.5% for support vector machines, 92.3% for nearest neighbors, 92.4% for both decision trees, and ensembles over all subjects’ data. Trajectory slopes of CORE vector magnitude and angle: m(|R|) and m(∠R) are the best-performing features, along with ensemble classifier with the highest accuracy of 95.3% and minimum computation time of 40 ms. The statistical significance of the results is validated with a p-value of less than 0.05. The proposed passive BCI scheme demonstrates a promising technique for online drowsiness detection using VPA along with sleep stage classification.

Exploration of illusory visual motion stimuli: An EEG-based brain-computer interface for practical assistive communication systems

This paper presents an illusory visual motion stimulus-based brain-computer interface (BCI). We aim to use the proposed system to enhance the motor imagery (MI) modality. Since motor imagery requires a long time for training, a stimulation method with external stimuli through the sensory system is an alternative method for increasing efficiency. The research is divided into two parts. First, we observed the visual motion illusion pattern based on brain topographic maps for the novel BCI modality. Second, we implemented the illusory visual motion stimulus-based BCI system. Arrow and moving-arrow patterns were used to modulate alpha rhythms at the visual and motor cortex. The arrow pattern had an average classification accuracy of approximately 78.5%. Additionally, illusory visual motion stimulus-based BCI systems are proposed using the proposed feature extraction and decision-making algorithm. This proposed BCI system can control the cursor moving in the left or right direction with the designed algorithm to create five commands for assistive communication. Ten volunteers participated in the experiment, and a brain-computer interface system with motor imagery and an illusory visual motion stimulus were used to compare efficiencies. The results showed that the proposed method achieved approximately 4% higher accuracy than motor imagery. The accuracy of the proposed illusory visual motion stimulus and algorithm was approximately 80.3%. Therefore, an illusory visual motion stimulus hybrid BCI system can be incorporated into the MI-based BCI system for beginner motor imagery. Based on the results, the proposed assistive communication system can be used to enhance communication in people with severe disabilities.

Toward a compact hybrid brain-computer interface (BCI): Performance evaluation of multi-class hybrid EEG-fNIRS BCIs with limited number of channels.

It has been demonstrated that the performance of typical unimodal brain-computer interfaces (BCIs) can be noticeably improved by combining two different BCI modalities. This so-called “hybrid BCI” technology has been studied for decades; however, hybrid BCIs that particularly combine electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) (hereafter referred to as hBCIs) have not been widely used in practical settings. One of the main reasons why hBCI systems are so unpopular is that their hardware is generally too bulky and complex. Therefore, to make hBCIs more appealing, it is necessary to implement a lightweight and compact hBCI system with minimal performance degradation. In this study, we investigated the feasibility of implementing a compact hBCI system with significantly less EEG channels and fNIRS source-detector (SD) pairs, but that can achieve a classification accuracy high enough to be used in practical BCI applications. EEG and fNIRS data were acquired while participants performed three different mental tasks consisting of mental arithmetic, right-hand motor imagery, and an idle state. Our analysis results showed that the three mental states could be classified with a fairly high classification accuracy of 77.6 ± 12.1% using an hBCI system with only two EEG channels and two fNIRS SD pairs.

Bio-Inspired Filter Banks for Frequency Recognition of SSVEP-Based Brain–Computer Interfaces

Brain-computer interfaces (BCIs) and their associated technologies have the potential to shape future forms of communication, control, and security. Specifically, the steady-state visual evoked potential (SSVEP) based BCIs have the advantages of better recognition accuracy, and higher information transfer rate (ITR) compared to other BCI modalities. To fully exploit the capabilities of such devices, it is necessary to understand the underlying biological features of SSVEPs and design the system considering their inherent characteristics. This paper introduces bio-inspired filter banks (BIFBs) for improved SSVEP frequency recognition. SSVEPs are frequency selective, subject-specific, and their power gets weaker as the frequency of the visual stimuli increases. Therefore, the gain and bandwidth of the filters are designed and tuned based on these characteristics while also incorporating harmonic SSVEP responses. The BIFBs are utilized in the feature extraction stage to increase the separability of classes. This method not only improves the recognition accuracy but also increases the total number of available commands in a BCI system by allowing the use of stimuli frequencies that elicit weak SSVEP responses. The BIFBs are promising particularly in the high-frequency band, which causes less visual fatigue. Hence, the proposed approach might enhance user comfort as well. The BIFB method is tested on two online benchmark datasets and outperforms the compared methods. The results show the potential of bio-inspired design, and the findings will be extended by including further SSVEP characteristics for future SSVEP based BCIs.

Binocular Phase-Coded Visual Stimuli for SSVEP-Based BCI

This paper presents a method of binocular visual stimulation for brain-computer interfaces (BCIs) based on steady-state visual evoked potentials (SSVEPs) using phase-coded symbols. The proposed method's emphasis is on a binocular phase-coded visual stimulus, which is based on the phase differences between the left- and right-eye stimuli, and a symbol detection and recognition procedure based on SSVEP response of the left and right occipital lobes of the user's scalp, where the SSVEP response is obtained as electroencephalography (EEG) signaling. The symbols are coded as phase differences and maintain the same frequency of the sine wave-modulated light provided to the user's left and right eyes as a binocular visual stimulation. Based on this method, a basic system setup is presented to explore the possibilities of binocular phase-coded visual stimuli for virtual or augmented reality applications, where the binocular visual stimulation was achieved by the specially designed head-mounted displays. Multiple visually coded targets are realized as eight different phase-coded binocular symbols and further evaluated as a random sequence of single targets, thus representing the situations in virtual or augmented reality, where multiple visually coded targets are present but not visualized to the user simultaneously within the same field of view. The offline results obtained from ten healthy subjects revealed that an average symbol recognition accuracy of 90.63% and an information transfer rate (ITR) of 70.55 bits/min were achieved for a symbol stimulation time of 2 s. The results of this paper demonstrate the feasibility of using binocular visual stimuli for SSVEP-based BCIs, where reasonable ITR is achieved using single-frequency binocular phase-coded symbols. The proposed method indicates the possibility of combining it with 3D wearable visualization technologies, such as binocular head-mounted displays (HMDs), in order to improve the intuitiveness of the interaction with more immersive user experience using BCI modalities.

Online classification of the near-infrared spectroscopy fast optical signal for brain-computer interfaces

Objective. The fast optical signal (FOS), measured with near-infrared spectroscopy (NIRS), has high temporal and competitive spatial resolution which provides an opportunity for a novel brain-computer interface modality. However, the reliability of the FOS has been debated due to its low signal-to-noise ratio. Approach. This study examined the feasibility of automatically classifying the prefrontal FOS response during a visual oddball task. FOS measurements were collected from 15 participants during 3 offline and 2 online sessions. Classification feedback was provided to participants during online sessions. The FOS classification algorithm discriminated between oddball and frequent responses. DC intensity and phase delay FOS measurements were classified individually with both support vector machine (SVM) and linear discriminant analysis (LDA) classifiers. The decisions of these classifiers were combined with a weighted majority vote. Fifteen-trial averages were selected for optimal classification results and the best feature types were found to be the number of zero crossing and the variance. Main results. FOS responses to oddball and frequent images were classified offline with an average balanced accuracy of 62 ± 5% and classified online with an averaged balanced accuracy of 63 ± 6% across all participants. Offline classification accuracies were significantly higher than chance for all participants. Online classification results were significantly above chance in both online sessions for 7 of 14 participants. Event-related potentials (ERPs) were classified using a similar algorithm with an average balanced accuracy of 77 ± 5%, confirming that the prefrontal neuronal response to the visual oddball task could be classified above the level required (>70%) for effective BCI communication. Significance. The FOS classification results demonstrated that automatic classification of the FOS is possible at above-chance levels, however, FOS classification accuracies did not reach the effective BCI communication threshold. Further FOS classification efforts should focus on investigating spectral features as well as adding measurement channels over the fronto-central and parietal areas of the brain.

EEG-Based Brain-Computer Interfaces for Communication and Rehabilitation of People with Motor Impairment: A Novel Approach of the 21 st Century.

People with severe neurological impairments face many challenges in sensorimotor functions and communication with the environment; therefore they have increased demand for advanced, adaptive and personalized rehabilitation. During the last several decades, numerous studies have developed brain–computer interfaces (BCIs) with the goals ranging from providing means of communication to functional rehabilitation. Here we review the research on non-invasive, electroencephalography (EEG)-based BCI systems for communication and rehabilitation. We focus on the approaches intended to help severely paralyzed and locked-in patients regain communication using three different BCI modalities: slow cortical potentials, sensorimotor rhythms and P300 potentials, as operational mechanisms. We also review BCI systems for restoration of motor function in patients with spinal cord injury and chronic stroke. We discuss the advantages and limitations of these approaches and the challenges that need to be addressed in the future.

Performance of Brain-Computer Interfacing Based on Tactile Selective Sensation and Motor Imagery.

A large proportion of users do not achieve adequate control using current non-invasive brain-computer interfaces (BCIs). This issue has being coined "BCI-Illiteracy" and is observed among different BCI modalities. Here, we compare the performance and the BCI-illiteracy rate of a tactile selective sensation (SS) and motor imagery (MI) BCI, for a large subject samples. We analyzed 80 experimental sessions from 57 subjects with two-class SS protocols. For SS, the group average performance was 79.8 ± 10.6%, with 43 out of the 57 subjects (75.4%) exceeding the 70% BCI-illiteracy threshold for left- and right-hand SS discrimination. When compared with previous results, this tactile BCI outperformed all other tactile BCIs currently available. We also analyzed 63 experimental sessions from 43 subjects with two-class MI BCI protocols, where the group average performance was 77.2 ± 13.3%, with 69.7% of the subjects exceeding the 70% performance threshold for left- and right-hand MI. For within-subject comparison, the 24 subjects who participated to both the SS and MI experiments, the BCI performance was superior with SS than MI especially in beta frequency band (p < 0.05), with enhanced R2 discriminative information in the somatosensory cortex for the SS modality. Both SS and MI showed a functional dissociation between lower alpha ([8 10] Hz) and upper alpha ([10 13] Hz) bands, with BCI performance significantly better in the upper alpha than the lower alpha (p < 0.05) band. In summary, we demonstrated that SS is a promising BCI modality with low BCI illiteracy issue and has great potential in practical applications reaching large population.

Polychromatic SSVEP stimuli with subtle flickering adapted to brain-display interactions

Objective. Interactive displays armed with natural user interfaces (NUIs) will likely lead the next breakthrough in consumer electronics, and brain-computer interfaces (BCIs) are often regarded as the ultimate NUI-enabling machines to respond to human emotions and mental states. Steady-state visual evoked potentials (SSVEPs) are a commonly used BCI modality due to the ease of detection and high information transfer rates. However, the presence of flickering stimuli may cause user discomfort and can even induce migraines and seizures. With the aim of designing visual stimuli that can be embedded into video images, this study developed a novel approach to induce detectable SSVEPs using a composition of red/green/blue flickering lights. Approach. Based on the opponent theory of colour vision, this study used 32 Hz/40 Hz rectangular red-green or red-blue LED light pulses with a 50% duty cycle, balanced/equal luminance and 0°/180° phase shifts as the stimulating light sources and tested their efficacy in producing SSVEP responses with high signal-to-noise ratios (SNRs) while reducing the perceived flickering sensation. Main results. The empirical results from ten healthy subjects showed that dual-colour lights flickering at 32 Hz/40 Hz with a 50% duty cycle and 180° phase shift achieved a greater than 90% detection accuracy with little or no flickering sensation. Significance. As a first step in developing an embedded SSVEP stimulus in commercial displays, this study provides a foundation for developing a combination of three primary colour flickering backlights with adjustable luminance proportions to create a subtle flickering polychromatic light that can elicit SSVEPs at the basic flickering frequency.

Towards Rehabilitation Robotics: Off-the-Shelf BCI Control of Anthropomorphic Robotic Arms.

Advances in neural interfaces have demonstrated remarkable results in the direction of replacing and restoring lost sensorimotor function in human patients. Noninvasive brain-computer interfaces (BCIs) are popular due to considerable advantages including simplicity, safety, and low cost, while recent advances aim at improving past technological and neurophysiological limitations. Taking into account the neurophysiological alterations of disabled individuals, investigating brain connectivity features for implementation of BCI control holds special importance. Off-the-shelf BCI systems are based on fast, reproducible detection of mental activity and can be implemented in neurorobotic applications. Moreover, social Human-Robot Interaction (HRI) is increasingly important in rehabilitation robotics development. In this paper, we present our progress and goals towards developing off-the-shelf BCI-controlled anthropomorphic robotic arms for assistive technologies and rehabilitation applications. We account for robotics development, BCI implementation, and qualitative assessment of HRI characteristics of the system. Furthermore, we present two illustrative experimental applications of the BCI-controlled arms, a study of motor imagery modalities on healthy individuals' BCI performance, and a pilot investigation on spinal cord injured patients' BCI control and brain connectivity. We discuss strengths and limitations of our design and propose further steps on development and neurophysiological study, including implementation of connectivity features as BCI modality.

How Much Control Is Enough? Influence of Unreliable Input on User Experience

Brain–computer interfaces (BCI) provide a valuable new input modality within human–computer interaction systems. However, like other body-based inputs such as gesture or gaze based systems, the system recognition of input commands is still far from perfect. This raises important questions, such as what level of control should such an interface be able to provide. What is the relationship between actual and perceived control? And in the case of applications for entertainment in which fun is an important part of user experience, should we even aim for the highest level of control, or is the optimum elsewhere? In this paper, we evaluate whether we can modulate the amount of control and if a game can be fun with less than perfect control. In the experiment users (n = 158) played a simple game in which a hamster has to be guided to the exit of a maze. The amount of control the user has over the hamster is varied. The variation of control through confusion matrices makes it possible to simulate the experience of using a BCI, while using the traditional keyboard for input. After each session the user completed a short questionnaire on user experience and perceived control. Analysis of the data showed that the perceived control of the user could largely be explained by the amount of control in the respective session. As expected, user frustration decreases with increasing control. Moreover, the results indicate that the relation between fun and control is not linear. Although at lower levels of control fun does increase with improved control, the level of fun drops just before perfect control is reached (with an optimum around 96%). This poses new insights for developers of games who want to incorporate some form of BCI or other modality with unreliable input in their game: for creating a fun game, unreliable input can be used to create a challenge for the user.

An online EEG BCI based on covert visuospatial attention in absence of exogenous stimulation

Objective. In this work we present—for the first time—the online operation of an electroencephalogram (EEG) brain–computer interface (BCI) system based on covert visuospatial attention (CVSA), without relying on any evoked responses. Electrophysiological correlates of pure top-down CVSA have only recently been proposed as a control signal for BCI. Such systems are expected to share the ease of use of stimulus-driven BCIs (e.g. P300, steady state visually evoked potential) with the autonomy afforded by decoding voluntary modulations of ongoing activity (e.g. motor imagery). Approach. Eight healthy subjects participated in the study. EEG signals were acquired with an active 64-channel system. The classification method was based on a time-dependent approach tuned to capture the most discriminant spectral features of the temporal evolution of attentional processes. The system was used by all subjects over two days without retraining, to verify its robustness and reliability. Main results. We report a mean online accuracy across the group of 70.6 ± 1.5%, and 88.8 ± 5.8% for the best subject. Half of the participants produced stable features over the entire duration of the study. Additionally, we explain drops in performance in subjects showing stable features in terms of known electrophysiological correlates of fatigue, suggesting the prospect of online monitoring of mental states in BCI systems. Significance. This work represents the first demonstration of the feasibility of an online EEG BCI based on CVSA. The results achieved suggest the CVSA BCI as a promising alternative to standard BCI modalities.

Auditory brain-computer interfaces (BCIs) and their practical applications

Brain-computer interfaces (BCIs) can provide a communication pathway for patients with severe neuromuscular diseases such as amyotrophic lateral sclerosis. Most current BCI systems rely upon the remaining visual function of individuals; however, patients with severe neuromuscular diseases may lose the ability to gaze at visual targets or to maintain gaze. Therefore, auditory-based BCI paradigms have been recently investigated as one of the alternative BCI modalities. This paper briefly reviews several studies focused on auditory BCI paradigms and discusses their prospects in terms of practical applications.

Spatial filters to detect steady-state visual evoked potentials elicited by high frequency stimulation: BCI application

Brain-computer interfaces (BCIs) based on steady-state visual evoked potentials (SSVEPs) require minimal user training and can offer higher information throughput compared to other BCI modalities. We focused on SSVEPs elicited by high-frequency stimuli (>30 Hz) because they cause minimal fatigue/annoyance and reduce the risk of inducing photoepileptic seizures. This paper presents an approach that analyzes electroencephalographic activity to automatically obtain the optimum spatial filter for detecting the SSVEP at a given stimulation frequency from a short signal where the stimulation is presented at intermittent periods interspersed with breaks. A vector space generated by sinusoidal signals at the stimulation frequency and harmonics is defined. The spatial filter coefficients result from maximizing the ratio between the energy of the spatially filtered signal and that of its orthogonal component with regard to the vector space. The spatial filters are customized for each BCI user through a short calibration procedure taking into account individual specificity. Our experiments on six subjects applying the spatial filters resulted in an average transfer rate ranging from 20.9 to 22.7 bits/min.