Passive Brain-computer Interface Research Articles

An Auditory BCI System for Assisting CRS-R Behavioral Assessment in Patients with Disorders of Consciousness.

The Coma Recovery Scale-Revised (CRS-R) is a consistent and sensitive behavioral assessment standard for disorders of consciousness (DOC) patients. However, the CRS-R has limitations due to its dependence on behavioral markers, which has led to a high rate of misdiagnosis. Brain-computer interfaces (BCIs), which directly detect brain activities without any behavioral expression, can be used to evaluate a patient’s state. In this study, we explored the application of BCIs in assisting CRS-R assessments of DOC patients. Specifically, an auditory passive EEG-based BCI system with an oddball paradigm was proposed to facilitate the evaluation of one item of the auditory function scale in the CRS-R – the auditory startle. The results obtained from five healthy subjects validated the efficacy of the BCI system. Nineteen DOC patients participated in the CRS-R and BCI assessments, of which three patients exhibited no responses in the CRS-R assessment but were responsive to auditory startle in the BCI assessment. These results revealed that a proportion of DOC patients who have no behavioral responses in the CRS-R assessment can generate neural responses, which can be detected by our BCI system. Therefore, the proposed BCI may provide more sensitive results than the CRS-R and thus assist CRS-R behavioral assessments.

Transfer learning with large-scale data in brain-computer interfaces.

Human variability in electroencephalogram (EEG) poses significant challenges for developing practical real-world applications of brain-computer interfaces (BCIs). The intuitive solution of collecting sufficient user-specific training/calibration data can be very labor-intensive and time-consuming, hindering the practicability of BCIs. To address this problem, transfer learning (TL), which leverages existing data from other sessions or subjects, has recently been adopted by the BCI community to build a BCI for a new user with limited calibration data. However, current TL approaches still require training/calibration data from each of conditions, which might be difficult or expensive to obtain. This study proposed a novel TL framework that could nearly eliminate requirement of subject-specific calibration data by leveraging large-scale data from other subjects. The efficacy of this method was validated in a passive BCI that was designed to detect neurocognitive lapses during driving. With the help of large-scale data, the proposed TL approach outperformed the within-subject approach while considerably reducing the amount of calibration data required for each individual (~1.5 min of data from each individual as opposed to a 90 min pilot session used in a standard within-subject approach). This demonstration might considerably facilitate the real-world applications of BCIs.

Privacy and brain-computer interfaces

Brain-Computer Interfaces (BCIs) interpret neural activity, applying it to the control of external devices. As BCIs approach market viability, ethical implications come under consideration. This paper identifies potential privacy disruptions. BCI literature is reviewed in order to identify a BCI typology likely to support a privacy analysis. The typology describes the active, reactive, passive and hybrid types of BCI and, where possible, includes examples that are further classified as existing, prospective or speculative. A review of privacy theory supports an analysis that juxtaposes privacy theory and BCI technologies. The analysis finds that while all four types of BCI have potential for disrupting privacy, disruptions are more likely to arise from the use of reactive, passive and hybrid BCIs. Limitations and directions for future research close the paper.

A passive brain-computer interface application for the mental workload assessment on professional air traffic controllers during realistic air traffic control tasks.

In the last decades, it has been a fast-growing concept in the neuroscience field. The passive brain-computer interface (p-BCI) systems allow to improve the human-machine interaction (HMI) in operational environments, by using the covert brain activity (eg, mental workload) of the operator. However, p-BCI technology could suffer from some practical issues when used outside the laboratories. In particular, one of the most important limitations is the necessity to recalibrate the p-BCI system each time before its use, to avoid a significant reduction of its reliability in the detection of the considered mental states. The objective of the proposed study was to provide an example of p-BCIs used to evaluate the users' mental workload in a real operational environment. For this purpose, through the facilities provided by the École Nationale de l'Aviation Civile of Toulouse (France), the cerebral activity of 12 professional air traffic control officers (ATCOs) has been recorded while performing high realistic air traffic management scenarios. By the analysis of the ATCOs' brain activity (electroencephalographic signal-EEG) and the subjective workload perception (instantaneous self-assessment) provided by both the examined ATCOs and external air traffic control experts, it has been possible to estimate and evaluate the variation of the mental workload under which the controllers were operating. The results showed (i) a high significant correlation between the neurophysiological and the subjective workload assessment, and (ii) a high reliability over time (up to a month) of the proposed algorithm that was also able to maintain high discrimination accuracies by using a low number of EEG electrodes (~3 EEG channels). In conclusion, the proposed methodology demonstrated the suitability of p-BCI systems in operational environments and the advantages of the neurophysiological measures with respect to the subjective ones.

Investigation of engagement of viewers in movie trailers using electroencephalography

Brain-computer interfaces (BCIs) have been focused on providing direct communications to the disabled. Recently, BCI researchers have expanded BCI applications to non-medical uses and categorized them as active BCI, reactive BCI, and passive BCI. Neurocinematics, a new application of reactive BCIs, aims to understand viewers’ cognitive and affective responses to movies from neural activity, providing more objective information than traditional subjective self-reports. However, studies on analytical indices for neurocinematics have verified their indices by comparisons with self-reports. To overcome this contradictory issue, we proposed using an independent psychophysical index to evaluate a neural engagement index (NEI). We made use of the secondary task reaction time (STRT), which measures participants’ engagement in a primary task by their reaction time to a secondary task; here, responding to a tactile stimulus was the secondary task and watching a movie trailer was the primary task. NEI was developed as changes in the difference between frontal beta and alpha activity of EEG. We evaluated movie trailers using NEI, STRT, and self-reports and found a significant correlation between STRT and NEI across trailers but no correlation between any of the self-report results and STRT or NEI. Our results suggest that NEI developed for neurocinematics may conform well with more objective psychophysical assessments but not with subjective self-reports.

Passive BCI based on drowsiness detection: an fNIRS study.

We use functional near-infrared spectroscopy (fNIRS) to discriminate the alert and drowsy states for a passive brain-computer interface (BCI). The passive brain signals for the drowsy state are acquired from the prefrontal and dorsolateral prefrontal cortex. The experiment is performed on 13 healthy subjects using a driving simulator, and their brain activity is recorded using a continuous-wave fNIRS system. Linear discriminant analysis (LDA) is employed for training and testing, using the data from the prefrontal, left- and right-dorsolateral prefrontal regions. For classification, eight features are tested: mean oxyhemoglobin, mean deoxyhemoglobin, skewness, kurtosis, signal slope, number of peaks, sum of peaks, and signal peak, in 0~5, 0~10, and 0~15 second time windows, respectively. The results show that the best performance for classification is achieved using mean oxyhemoglobin, the signal peak, and the sum of peaks as features. The average accuracies in the right dorsolateral prefrontal cortex (83.1, 83.4 and 84.9% in the 0~5, 0~10 and 0~15 second time windows, respectively) show that the proposed method has an effective utility for detection of drowsiness for a passive BCI.

Real-time state estimation in a flight simulator using fNIRS.

Working memory is a key executive function for flying an aircraft. This function is particularly critical when pilots have to recall series of air traffic control instructions. However, working memory limitations may jeopardize flight safety. Since the functional near-infrared spectroscopy (fNIRS) method seems promising for assessing working memory load, our objective is to implement an on-line fNIRS-based inference system that integrates two complementary estimators. The first estimator is a real-time state estimation MACD-based algorithm dedicated to identifying the pilot’s instantaneous mental state (not-on-task vs. on-task). It does not require a calibration process to perform its estimation. The second estimator is an on-line SVM-based classifier that is able to discriminate task difficulty (low working memory load vs. high working memory load). These two estimators were tested with 19 pilots who were placed in a realistic flight simulator and were asked to recall air traffic control instructions. We found that the estimated pilot’s mental state matched significantly better than chance with the pilot’s real state (62% global accuracy, 58% specificity, and 72% sensitivity). The second estimator, dedicated to assessing single trial working memory loads, led to 80% classification accuracy, 72% specificity, and 89% sensitivity. These two estimators establish reusable blocks for further fNIRS-based passive brain computer interface development.

Electrode replacement does not affect classification accuracy in dual-session use of a passive brain-computer interface for assessing cognitive workload.

The passive brain-computer interface (pBCI) framework has been shown to be a very promising construct for assessing cognitive and affective state in both individuals and teams. There is a growing body of work that focuses on solving the challenges of transitioning pBCI systems from the research laboratory environment to practical, everyday use. An interesting issue is what impact methodological variability may have on the ability to reliably identify (neuro)physiological patterns that are useful for state assessment. This work aimed at quantifying the effects of methodological variability in a pBCI design for detecting changes in cognitive workload. Specific focus was directed toward the effects of replacing electrodes over dual sessions (thus inducing changes in placement, electromechanical properties, and/or impedance between the electrode and skin surface) on the accuracy of several machine learning approaches in a binary classification problem. In investigating these methodological variables, it was determined that the removal and replacement of the electrode suite between sessions does not impact the accuracy of a number of learning approaches when trained on one session and tested on a second. This finding was confirmed by comparing to a control group for which the electrode suite was not replaced between sessions. This result suggests that sensors (both neurological and peripheral) may be removed and replaced over the course of many interactions with a pBCI system without affecting its performance. Future work on multi-session and multi-day pBCI system use should seek to replicate this (lack of) effect between sessions in other tasks, temporal time courses, and data analytic approaches while also focusing on non-stationarity and variable classification performance due to intrinsic factors.

Cognitive state monitoring and the design of adaptive instruction in digital environments: lessons learned from cognitive workload assessment using a passive brain-computer interface approach.

According to Cognitive Load Theory (CLT), one of the crucial factors for successful learning is the type and amount of working-memory load (WML) learners experience while studying instructional materials. Optimal learning conditions are characterized by providing challenges for learners without inducing cognitive over- or underload. Thus, presenting instruction in a way that WML is constantly held within an optimal range with regard to learners' working-memory capacity might be a good method to provide these optimal conditions. The current paper elaborates how digital learning environments, which achieve this goal can be developed by combining approaches from Cognitive Psychology, Neuroscience, and Computer Science. One of the biggest obstacles that needs to be overcome is the lack of an unobtrusive method of continuously assessing learners' WML in real-time. We propose to solve this problem by applying passive Brain-Computer Interface (BCI) approaches to realistic learning scenarios in digital environments. In this paper we discuss the methodological and theoretical prospects and pitfalls of this approach based on results from the literature and from our own research. We present a strategy on how several inherent challenges of applying BCIs to WML and learning can be met by refining the psychological constructs behind WML, by exploring their neural signatures, by using these insights for sophisticated task designs, and by optimizing algorithms for analyzing electroencephalography (EEG) data. Based on this strategy we applied machine-learning algorithms for cross-task classifications of different levels of WML to tasks that involve studying realistic instructional materials. We obtained very promising results that yield several recommendations for future work.

Artificial Neural Network classification of operator workload with an assessment of time variation and noise-enhancement to increase performance.

Workload classification—the determination of whether a human operator is in a high or low workload state to allow their working environment to be optimized—is an emerging application of passive Brain-Computer Interface (BCI) systems. Practical systems must not only accurately detect the current workload state, but also have good temporal performance: requiring little time to set up and train the classifier, and ensuring that the reported performance level is consistent and predictable over time. This paper investigates the temporal performance of an Artificial Neural Network based classification system. For networks trained on little EEG data good classification accuracies (86%) are achieved over very short time frames, but substantial decreases in accuracy are found as the time gap between the network training and the actual use is increased. Noise-enhanced processing, where artificially generated noise is deliberately added to the testing signals, is investigated as a potential technique to mitigate this degradation without requiring the network to be re-trained using more data. Small stochastic resonance effects are demonstrated whereby the classification process gets better in the presence of more noise. The effect is small and does not eliminate the need for re-training, but it is consistent, and this is the first demonstration of such effects for non-evoked/free-running EEG signals suitable for passive BCI.

A review of brain-computer interface games and an opinion survey from researchers, developers and users.

In recent years, research on Brain-Computer Interface (BCI) technology for healthy users has attracted considerable interest, and BCI games are especially popular. This study reviews the current status of, and describes future directions, in the field of BCI games. To this end, we conducted a literature search and found that BCI control paradigms using electroencephalographic signals (motor imagery, P300, steady state visual evoked potential and passive approach reading mental state) have been the primary focus of research. We also conducted a survey of nearly three hundred participants that included researchers, game developers and users around the world. From this survey, we found that all three groups (researchers, developers and users) agreed on the significant influence and applicability of BCI and BCI games, and they all selected prostheses, rehabilitation and games as the most promising BCI applications. User and developer groups tended to give low priority to passive BCI and the whole head sensor array. Developers gave higher priorities to “the easiness of playing” and the “development platform” as important elements for BCI games and the market. Based on our assessment, we discuss the critical point at which BCI games will be able to progress from their current stage to widespread marketing to consumers. In conclusion, we propose three critical elements important for expansion of the BCI game market: standards, gameplay and appropriate integration.

Comparison of Sensor Selection Mechanisms for an ERP-Based Brain-Computer Interface

A major barrier for a broad applicability of brain-computer interfaces (BCIs) based on electroencephalography (EEG) is the large number of EEG sensor electrodes typically used. The necessity for this results from the fact that the relevant information for the BCI is often spread over the scalp in complex patterns that differ depending on subjects and application scenarios. Recently, a number of methods have been proposed to determine an individual optimal sensor selection. These methods have, however, rarely been compared against each other or against any type of baseline. In this paper, we review several selection approaches and propose one additional selection criterion based on the evaluation of the performance of a BCI system using a reduced set of sensors. We evaluate the methods in the context of a passive BCI system that is designed to detect a P300 event-related potential and compare the performance of the methods against randomly generated sensor constellations. For a realistic estimation of the reduced system's performance we transfer sensor constellations found on one experimental session to a different session for evaluation. We identified notable (and unanticipated) differences among the methods and could demonstrate that the best method in our setup is able to reduce the required number of sensors considerably. Though our application focuses on EEG data, all presented algorithms and evaluation schemes can be transferred to any binary classification task on sensor arrays.

Classifying visuomotor workload in a driving simulator using subject specific spatial brain patterns

A passive Brain Computer Interface (BCI) is a system that responds to the spontaneously produced brain activity of its user and could be used to develop interactive task support. A human-machine system that could benefit from brain-based task support is the driver-car interaction system. To investigate the feasibility of such a system to detect changes in visuomotor workload, 34 drivers were exposed to several levels of driving demand in a driving simulator. Driving demand was manipulated by varying driving speed and by asking the drivers to comply to individually set lane keeping performance targets. Differences in the individual driver's workload levels were classified by applying the Common Spatial Pattern (CSP) and Fisher's linear discriminant analysis to frequency filtered electroencephalogram (EEG) data during an off line classification study. Several frequency ranges, EEG cap configurations, and condition pairs were explored. It was found that classifications were most accurate when based on high frequencies, larger electrode sets, and the frontal electrodes. Depending on these factors, classification accuracies across participants reached about 95% on average. The association between high accuracies and high frequencies suggests that part of the underlying information did not originate directly from neuronal activity. Nonetheless, average classification accuracies up to 75–80% were obtained from the lower EEG ranges that are likely to reflect neuronal activity. For a system designer, this implies that a passive BCI system may use several frequency ranges for workload classifications.

Designing a passive brain computer interface using real time classification of functional near-infrared spectroscopy

Passive brain-computer interfaces consider brain activity as an additional source of information, to augment and adapt the interface instead of controlling it. We have developed a software system that allows for real time brain signal analysis and machine learning classification of affective and workload states measured with functional near-infrared spectroscopy (fNIRS) called the online fNIRS analysis and classification (OFAC). Our system reproduces successful offline procedures, adapting them for real-time input to a user interface. Our first evaluation compares a previous offline analysis with our online analysis. While results show an accuracy decrease, they are outweighed by the new ability of interface adaptation. The second study demonstrates OFAC's online features through real-time classification of two tasks, and interface adaptation according to the predicted task. Accuracy averaged over 85%. We have created the first working real time passive BCI using fNIRS, opening the door to build adaptive user interfaces.

Context-aware brain–computer interfaces: exploring the information space of user, technical system and environment**Parts of this work have already been presented in non-journal publications. This will be indicated specifically by appropriate references in the text.

Brain–computer interface (BCI) systems are usually applied in highly controlled environments such as research laboratories or clinical setups. However, many BCI-based applications are implemented in more complex environments. For example, patients might want to use a BCI system at home, and users without disabilities could benefit from BCI systems in special working environments. In these contexts, it might be more difficult to reliably infer information about brain activity, because many intervening factors add up and disturb the BCI feature space. One solution for this problem would be adding context awareness to the system. We propose to augment the available information space with additional channels carrying information about the user state, the environment and the technical system. In particular, passive BCI systems seem to be capable of adding highly relevant context information—otherwise covert aspects of user state. In this paper, we present a theoretical framework based on general human–machine system research for adding context awareness to a BCI system. Building on that, we present results from a study on a passive BCI, which allows access to the covert aspect of user state related to the perceived loss of control. This study is a proof of concept and demonstrates that context awareness could beneficially be implemented in and combined with a BCI system or a general human–machine system. The EEG data from this experiment are available for public download at www.phypa.org.