Passive Brain-computer Interface Research Articles

Enhancing learning experiences: EEG-based passive BCI system adapts learning speed to cognitive load in real-time, with motivation as catalyst.

Computer-based learning has gained popularity in recent years, providing learners greater flexibility and freedom. However, these learning environments do not consider the learner's mental state in real-time, resulting in less optimized learning experiences. This research aimed to explore the effect on the learning experience of a novel EEG-based Brain-Computer Interface (BCI) that adjusts the speed of information presentation in real-time during a learning task according to the learner's cognitive load. We also explored how motivation moderated these effects. In accordance with three experimental groups (non-adaptive, adaptive, and adaptive with motivation), participants performed a calibration task (n-back), followed by a memory-based learning task concerning astrological constellations. Learning gains were assessed based on performance on the learning task. Self-perceived mental workload, cognitive absorption and satisfaction were assessed using a post-test questionnaire. Between-group analyses using Mann-Whitney tests suggested that combining BCI and motivational factors led to more significant learning gains and an improved learning experience. No significant difference existed between the BCI without motivational factor and regular non-adaptive interface for overall learning gains, self-perceived mental workload, and cognitive absorption. However, participants who undertook the experiment with an imposed learning pace reported higher overall satisfaction with their learning experience and a higher level of temporal stress. Our findings suggest BCI's potential applicability and feasibility in improving memorization-based learning experiences. Further work should seek to optimize the BCI adaptive index and explore generalizability to other learning contexts.

Advancing passive BCIs: a feasibility study of two temporal derivative features and effect size-based feature selection in continuous online EEG-based machine error detection.

The emerging integration of Brain-Computer Interfaces (BCIs) in human-robot collaboration holds promise for dynamic adaptive interaction. The use of electroencephalogram (EEG)-measured error-related potentials (ErrPs) for online error detection in assistive devices offers a practical method for improving the reliability of such devices. However, continuous online error detection faces challenges such as developing efficient and lightweight classification techniques for quick predictions, reducing false alarms from artifacts, and dealing with the non-stationarity of EEG signals. Further research is essential to address the complexities of continuous classification in online sessions. With this study, we demonstrated a comprehensive approach for continuous online EEG-based machine error detection, which emerged as the winner of a competition at the 32nd International Joint Conference on Artificial Intelligence. The competition consisted of two stages: an offline stage for model development using pre-recorded, labeled EEG data, and an online stage 3 months after the offline stage, where these models were tested live on continuously streamed EEG data to detect errors in orthosis movements in real time. Our approach incorporates two temporal-derivative features with an effect size-based feature selection technique for model training, together with a lightweight noise filtering method for online sessions without recalibration of the model. The model trained in the offline stage not only resulted in a high average cross-validation accuracy of 89.9% across all participants, but also demonstrated remarkable performance during the online session 3 months after the initial data collection without further calibration, maintaining a low overall false alarm rate of 1.7% and swift response capabilities. Our research makes two significant contributions to the field. Firstly, it demonstrates the feasibility of integrating two temporal derivative features with an effect size-based feature selection strategy, particularly in online EEG-based BCIs. Secondly, our work introduces an innovative approach designed for continuous online error prediction, which includes a straightforward noise rejection technique to reduce false alarms. This study serves as a feasibility investigation into a methodology for seamless error detection that promises to transform practical applications in the domain of neuroadaptive technology and human-robot interaction.

Ecological decoding of visual aesthetic preference with oscillatory electroencephalogram features-A mini-review.

In today's digital information age, human exposure to visual artifacts has reached an unprecedented quasi-omnipresence. Some of these cultural artifacts are elevated to the status of artworks which indicates a special appreciation of these objects. For many persons, the perception of such artworks coincides with aesthetic experiences (AE) that can positively affect health and wellbeing. AEs are composed of complex cognitive and affective mental and physiological states. More profound scientific understanding of the neural dynamics behind AEs would allow the development of passive Brain-Computer-Interfaces (BCI) that offer personalized art presentation to improve AE without the necessity of explicit user feedback. However, previous empirical research in visual neuroaesthetics predominantly investigated functional Magnetic Resonance Imaging and Event-Related-Potentials correlates of AE in unnaturalistic laboratory conditions which might not be the best features for practical neuroaesthetic BCIs. Furthermore, AE has, until recently, largely been framed as the experience of beauty or pleasantness. Yet, these concepts do not encompass all types of AE. Thus, the scope of these concepts is too narrow to allow personalized and optimal art experience across individuals and cultures. This narrative mini-review summarizes the state-of-the-art in oscillatory Electroencephalography (EEG) based visual neuroaesthetics and paints a road map toward the development of ecologically valid neuroaesthetic passive BCI systems that could optimize AEs, as well as their beneficial consequences. We detail reported oscillatory EEG correlates of AEs, as well as machine learning approaches to classify AE. We also highlight current limitations in neuroaesthetics and suggest future directions to improve EEG decoding of AE.

EEG-based hierarchical classification of level of demand and modality of auditory and visual sensory processing

Objective. To date, most research on electroencephalography (EEG)-based mental workload detection for passive brain-computer interface (pBCI) applications has focused on identifying the overall level of cognitive resources required, such as whether the workload is high or low. We propose, however, that being able to determine the specific type of cognitive resources being used, such as visual or auditory, would also be useful. This would enable the pBCI to take more appropriate action to reduce the overall level of cognitive demand on the user. For example, if a high level of workload was detected and it is determined that the user is primarily engaged in visual information processing, then the pBCI could cause some information to be presented aurally instead. In our previous work we showed that EEG could be used to differentiate visual from auditory processing tasks when the level of processing is high, but the two modalities could not be distinguished when the level of cognitive processing demand was very low. The current study aims to build on this work and move toward the overall objective of developing a pBCI that is capable of predicting both the level and the type of cognitive resources being used. Approach. Fifteen individuals undertook carefully designed visual and auditory tasks while their EEG data was being recorded. In this study, we incorporated a more diverse range of sensory processing conditions including not only single-modality conditions (i.e. those requiring one of either visual or auditory processing) as in our previous study, but also dual-modality conditions (i.e. those requiring both visual and auditory processing) and no-task/baseline conditions (i.e. when the individual is not engaged in either visual or auditory processing). Main results. Using regularized linear discriminant analysis within a hierarchical classification algorithm, the overall cognitive demand was predicted with an accuracy of more than 86%, while the presence or absence of visual and auditory sensory processing were each predicted with an accuracy of approximately 70%. Significance. The findings support the feasibility of establishing a pBCI that can determine both the level and type of attentional resources required by the user at any given moment. This pBCI could assist in enhancing safety in hazardous jobs by triggering the most effective and efficient adaptation strategies when high workload conditions are detected.

Human–Computer Interaction Multi-Task Modeling Based on Implicit Intent EEG Decoding

In the short term, a fully autonomous level of machine intelligence cannot be achieved. Humans are still an important part of HCI systems, and intelligent systems should be able to “feel” and “predict” human intentions in order to achieve dynamic coordination between humans and machines. Intent recognition is very important to improve the accuracy and efficiency of the HCI system. However, it is far from enough to focus only on explicit intent. There is a lot of vague and hidden implicit intent in the process of human–computer interaction. Based on passive brain–computer interface (pBCI) technology, this paper proposes a method to integrate humans into HCI systems naturally, which is to establish an intent-based HCI model and automatically recognize the implicit intent according to human EEG signals. In view of the existing problems of few divisible patterns and low efficiency of implicit intent recognition, this paper finally proves that EEG can be used as the basis for judging human implicit intent through extracting multi-task intention, carrying out experiments, and constructing algorithmic models. The CSP + SVM algorithm model can effectively improve the EEG decoding performance of implicit intent in HCI, and the effectiveness of the CSP algorithm on intention feature extraction is further verified by combining 3D space visualization. The translation of implicit intent information is of significance for the study of intent-based HCI models, the development of HCI systems, and the improvement of human–machine collaboration efficiency.

Wearable brain-computer interface technology and its application

Wearable Brain-Computer Interfaces (BCIs) signify a critical evolution in human-machine communication, driven by the convergence of neuroscience, engineering, and information technology. With applications that span across industrial, medical, and recreational domains, BCIs hold potential to redefine our interaction with the technological landscape. This manuscript elucidates this transformative juncture, bifurcating into passive and active BCIs. In passive BCIs, innovations leveraging Virtual Reality (VR) and Augmented Reality (AR) are delineated, progress has been demonstrated in the classification of passively induced emotional signals, highlighting the emergence of hands-free control systems like quadcopter control and industrial inspections. The discourse on active BCIs unveils advancements such as real-time handwriting and speech decoding, driver drowsiness detection, and emotion recognition, aided by machine learning techniques. Despite groundbreaking progress, challenges in algorithm optimization, adaptability, multimodal signal complexity, and ethics persist. Future directions emphasize the potential of deep learning and multimode input signals collaboration. The manuscript underscores the societal implications, particularly in rehabilitation, communication, and entertainment. The review, therefore, serves as both an appraisal and a roadmap for the burgeoning field of wearable BCIs, underlining its role as a pathway to enhance human capabilities and quality of life.

Unsupervised Detection of Covariate Shift Due to Changes in EEG Headset Position: Towards an Effective Out-of-Lab Use of Passive Brain–Computer Interface

In the field of passive Brain–computer Interfaces (BCI), the need to develop systems that require rapid setup, suitable for use outside of laboratories is a fundamental challenge, especially now, that the market is flooded with novel EEG headsets with a good quality. However, the lack of control in operational conditions can compromise the performance of the machine learning model behind the BCI system. First, this study focuses on evaluating the performance loss of the BCI system, induced by a different positioning of the EEG headset (and of course sensors), so generating a variation in the control features used to calibrate the machine learning algorithm. This phenomenon is called covariate shift. Detecting covariate shift occurrences in advance allows for preventive measures, such as informing the user to adjust the position of the headset or applying specific corrections in new coming data. We used in this study an unsupervised Machine Learning model, the Isolation Forest, to detect covariate shift occurrence in new coming data. We tested the method on two different datasets, one in a controlled setting (9 participants), and the other in a more realistic setting (10 participants). In the controlled dataset, we simulated the movement of the EEG cap using different channel and reference configurations. For each test configuration, we selected a set of electrodes near the control electrodes. Regarding the realistic dataset, we aimed to simulate the use of the cap outside the laboratory, mimicking the removal and repositioning of the cap by a non-expert user. In both datasets, we recorded multiple test sessions for each configuration while executing a set of Workload tasks. The results obtained using the Isolation Forest model allowed the identification of covariate shift in the data, even with a 15-s recording sample. Moreover, the results showed a strong and significant negative correlation between the percentage of covariate shift detected by the method, and the accuracy of the passive BCI system (p-value < 0.01). This novel approach opens new perspectives for developing more robust and flexible BCI systems, with the potential to move these technologies towards out-of-the-lab use, without the need for supervision for use by a non-expert user.

Assessment of mental workload across cognitive tasks using a passive brain-computer interface based on mean negative theta-band amplitudes.

Brain-computer interfaces (BCI) can provide real-time and continuous assessments of mental workload in different scenarios, which can subsequently be used to optimize human-computer interaction. However, assessment of mental workload is complicated by the task-dependent nature of the underlying neural signals. Thus, classifiers trained on data from one task do not generalize well to other tasks. Previous attempts at classifying mental workload across different cognitive tasks have therefore only been partially successful. Here we introduce a novel algorithm to extract frontal theta oscillations from electroencephalographic (EEG) recordings of brain activity and show that it can be used to detect mental workload across different cognitive tasks. We use a published data set that investigated subject dependent task transfer, based on Filter Bank Common Spatial Patterns. After testing, our approach enables a binary classification of mental workload with performances of 92.00 and 92.35%, respectively for either low or high workload vs. an initial no workload condition, with significantly better results than those of the previous approach. It, nevertheless, does not perform beyond chance level when comparing high vs. low workload conditions. Also, when an independent component analysis was done first with the data (and before any additional preprocessing procedure), even though we achieved more stable classification results above chance level across all tasks, it did not perform better than the previous approach. These mixed results illustrate that while the proposed algorithm cannot replace previous general-purpose classification methods, it may outperform state-of-the-art algorithms in specific (workload) comparisons.

Real-time assessment of hypnotic depth, using an EEG-based brain-computer interface: a preliminary study

ObjectiveHypnosis can be an effective treatment for many conditions, and there have been attempts to develop instrumental approaches to continuously monitor hypnotic state level (“depth”). However, there is no method that addresses the individual variability of electrophysiological hypnotic correlates. We explore the possibility of using an EEG-based passive brain-computer interface (pBCI) for real-time, individualised estimation of the hypnosis deepening process.ResultsThe wakefulness and deep hypnosis intervals were manually defined and labelled in 27 electroencephalographic (EEG) recordings obtained from eight outpatients after hypnosis sessions. Spectral analysis showed that EEG correlates of deep hypnosis were relatively stable in each patient throughout the treatment but varied between patients. Data from each first session was used to train classification models to continuously assess deep hypnosis probability in subsequent sessions. Models trained using four frequency bands (1.5–45, 1.5–8, 1.5–14, and 4–15 Hz) showed accuracy mostly exceeding 85% in a 10-fold cross-validation. Real-time classification accuracy was also acceptable, so at least one of the four bands yielded results exceeding 74% in any session. The best results averaged across all sessions were obtained using 1.5–14 and 4–15 Hz, with an accuracy of 82%. The revealed issues are also discussed.

A spectral-ensemble deep random vector functional link network for passive brain–computer interface

A spectral-ensemble deep random vector functional link network for passive brain–computer interface

Machine learning approach for early onset dementia neurobiomarker using EEG network topology features.

Modern neurotechnology research employing state-of-the-art machine learning algorithms within the so-called "AI for social good" domain contributes to improving the well-being of individuals with a disability. Using digital health technologies, home-based self-diagnostics, or cognitive decline managing approaches with neuro-biomarker feedback may be helpful for older adults to remain independent and improve their wellbeing. We report research results on early-onset dementia neuro-biomarkers to scrutinize cognitive-behavioral intervention management and digital non-pharmacological therapies. We present an empirical task in the EEG-based passive brain-computer interface application framework to assess working memory decline for forecasting a mild cognitive impairment. The EEG responses are analyzed in a framework of a network neuroscience technique applied to EEG time series for evaluation and to confirm the initial hypothesis of possible ML application modeling mild cognitive impairment prediction. We report findings from a pilot study group in Poland for a cognitive decline prediction. We utilize two emotional working memory tasks by analyzing EEG responses to facial emotions reproduced in short videos. A reminiscent interior image oddball task is also employed to validate the proposed methodology further. The proposed three experimental tasks in the current pilot study showcase the critical utilization of artificial intelligence for early-onset dementia prognosis in older adults.

Driving drowsiness detection using spectral signatures of EEG-based neurophysiology.

Introduction: Drowsy driving is a significant factor causing dire road crashes and casualties around the world. Detecting it earlier and more effectively can significantly reduce the lethal aftereffects and increase road safety. As physiological conditions originate from the human brain, so neurophysiological signatures in drowsy and alert states may be investigated for this purpose. In this preface, A passive brain-computer interface (pBCI) scheme using multichannel electroencephalography (EEG) brain signals is developed for spatially localized and accurate detection of human drowsiness during driving tasks. Methods: This pBCI modality acquired electrophysiological patterns of 12 healthy subjects from the prefrontal (PFC), frontal (FC), and occipital cortices (OC) of the brain. Neurological states are recorded using six EEG channels spread over the right and left hemispheres in the PFC, FC, and OC of the sleep-deprived subjects during simulated driving tasks. In post-hoc analysis, spectral signatures of the δ, θ, α, and β rhythms are extracted in terms of spectral band powers and their ratios with a temporal correlation over the complete span of the experiment. Minimum redundancy maximum relevance, Chi-square, and ReliefF feature selection methods are used and aggregated with a Z-score based approach for global feature ranking. The extracted drowsiness attributes are classified using decision trees, discriminant analysis, logistic regression, naïve Bayes, support vector machines, k-nearest neighbors, and ensemble classifiers. The binary classification results are reported with confusion matrix-based performance assessment metrics. Results: In inter-classifier comparison, the optimized ensemble model achieved the best results of drowsiness classification with 85.6% accuracy and precision, 89.7% recall, 87.6% F1-score, 80% specificity, 70.3% Matthews correlation coefficient, 70.2% Cohen's kappa score, and 91% area under the receiver operating characteristic curve with 76-ms execution time. In inter-channel comparison, the best results were obtained at the F8 electrode position in the right FC of the brain. The significance of all the results was validated with a p-value of less than 0.05 using statistical hypothesis testing methods. Conclusions: The proposed scheme has achieved better results for driving drowsiness detection with the accomplishment of multiple objectives. The predictor importance approach has reduced the feature extraction cost and computational complexity is minimized with the use of conventional machine learning classifiers resulting in low-cost hardware and software requirements. The channel selection approach has spatially localized the most promising brain region for drowsiness detection with only a single EEG channel (F8) which reduces the physical intrusiveness in normal driving operation. This pBCI scheme has a good potential for practical applications requiring earlier, more accurate, and less disruptive drowsiness detection using the spectral information of EEG biosignals.

Open multi-session and multi-task EEG cognitive Dataset for passive brain-computer Interface Applications

Brain-Computer Interfaces and especially passive Brain-Computer interfaces (pBCI), with their ability to estimate and monitor user mental states, are receiving increasing attention from both the fundamental research and the applied research and development communities. Testing new pipelines and benchmarking classifiers and feature extraction algorithms is central to further research within this domain. Unfortunately, data sharing in pBCI research is still scarce. The COG-BCI database encompasses the recordings of 29 participants over 3 separate sessions with 4 different tasks (MATB, N-Back, PVT, Flanker) designed to elicit different mental states, for a total of over 100 hours of open EEG data. This dataset was validated on a subjective, behavioral and physiological level, to ensure its usefulness to the pBCI community. Furthermore, a proof of concept is given with an example of mental workload estimation pipeline and results, to ensure that the data can be used for the design and evaluation of pBCI pipelines. This body of work presents a large effort to promote the use of pBCIs in an open science framework.

EEG-based detection of modality-specific visual and auditory sensory processing

Objective. A passive brain–computer interface (pBCI) is a system that enhances a human–machine interaction by monitoring the mental state of the user and, based on this implicit information, making appropriate modifications to the interaction. Key to the development of such a system is the ability to reliably detect the mental state of interest via neural signals. Many different mental states have been investigated, including fatigue, attention and various emotions, however one of the most commonly studied states is mental workload, i.e. the amount of attentional resources required to perform a task. The emphasis of mental workload studies to date has been almost exclusively on detecting and predicting the ‘level’ of cognitive resources required (e.g. high vs. low), but we argue that having information regarding the specific ‘type’ of resources (e.g. visual or auditory) would allow the pBCI to apply more suitable adaption techniques than would be possible knowing just the overall workload level. Approach. 15 participants performed carefully designed visual and auditory tasks while electroencephalography (EEG) data was recorded. The tasks were designed to be as similar as possible to one another except for the type of attentional resources required. The tasks were performed at two different levels of demand. Using traditional machine learning algorithms, we investigated, firstly, if EEG can be used to distinguish between auditory and visual processing tasks and, secondly, what effect level of sensory processing demand has on the ability to distinguish between auditory and visual processing tasks. Main results. The results show that at the high level of demand, the auditory vs. visual processing tasks could be distinguished with an accuracy of 77.1% on average. However, in the low demand condition in this experiment, the tasks were not classified with an accuracy exceeding chance. Significance. These results support the feasibility of developing a pBCI for detecting not only the level, but also the type, of attentional resources being required of the user at a given time. Further research is required to determine if there is a threshold of demand under which the type of sensory processing cannot be detected, but even if that is the case, these results are still promising since it is the high end of demand that is of most concern in safety critical scenarios. Such a BCI could help improve safety in high risk occupations by initiating the most effective and efficient possible adaptation strategies when high workload conditions are detected.

Development of a Computer-Aided Education System Inspired by Face-to-Face Learning by Incorporating EEG-Based Neurofeedback Into Online Video Lectures

Owing to the rapid development of information and communication technologies, online or mobile learning content is widely available on the Internet. Unlike traditional face-to-face learning, online learning exhibits a critical limitation: real-time interactions between learners and teachers are generally not feasible in online learning. To overcome this issue, we implemented an online learning system based on electroencephalography (EEG)-based passive brain-computer interface technology referred to as the “adaptive neuro-learning system (ANLS).” It monitors the current mental states of learners seamlessly using EEG signals. Then, it adaptively provides natural and interactive video feedback rather than simple alarms or pop quizzes following the current mental conditions of a learner. In this article, a total of 60 university students were assigned randomly to one of four groups: two experimental groups, for which either ANLS based on attention state estimation or ANLS based on both attention and comprehension states estimation was tested, and two control groups, the students in which were taught using either the conventional online lecture without feedback or an online course with randomized video feedbacks. Each member of these groups attended a 53 min open courseware video lecture. Then, the educational effects of the proposed system were evaluated quantitatively via a written examination. Our results revealed a significantly higher learning performance for the experimental group (average test score of the experimental groups = 83.83 and that of the control groups = 56.67), demonstrating the feasibility of the proposed education strategy.

A survey on encoding and decoding technology of non-invasive brain-computer interface

A survey on encoding and decoding technology of non-invasive brain-computer interface

A hybrid neuro-experimental decision support system to classify overconfidence and performance in a simulated bubble using a passive BCI

Significant advancements in brain-computer interfaces (BCIs) can lead to the development of enhanced decision-making platforms. Irrational behavior generating potential negative consequences can be better calibrated using experimental approaches. One such example is asset bubbles. Experimental methods using neuroimaging equipment can improve understanding of the phenomenon, while advancements in artificial intelligence can be used to classify and predict behavior based on the cognitive neurodynamics at play. Towards this goal, we ran an EEG-based experiment where subjects traded in a risky environment simulating the boom and bust of a financial bubble. We used blind source separation and empirical mode decomposition on EEG neural data to estimate approximate entropy as a metric of brain activity, which was subsequently used as feature vector input into various machine learning classifiers. We thus develop a decision support system to classify subjects according to overconfidence and trading performance using a hybrid classification methodology. We report average accuracy ratios ranging between 80 and 85%, using classifiers such as linear discriminants, decision trees, support vector machines, Naïve Bayes, multi-layer perceptron, and an artificial neural network. Not only does the hybrid method we employ produce high classification accuracy ratios in a cognitively demanding task of financial decision-making, but it can also be used in the classification of motor imagery tasks that can be coupled with financial decisions in the future. We contribute to the literature in neurofinance from an empirical perspective, while also providing a technical framework that can act as a basis for developing expert BCI systems.

Evaluation of a New Lightweight EEG Technology for Translational Applications of Passive Brain-Computer Interfaces.

Technologies like passive brain-computer interfaces (BCI) can enhance human-machine interaction. Anyhow, there are still shortcomings in terms of easiness of use, reliability, and generalizability that prevent passive-BCI from entering real-life situations. The current work aimed to technologically and methodologically design a new gel-free passive-BCI system for out-of-the-lab employment. The choice of the water-based electrodes and the design of a new lightweight headset met the need for easy-to-wear, comfortable, and highly acceptable technology. The proposed system showed high reliability in both laboratory and realistic settings, performing not significantly different from the gold standard based on gel electrodes. In both cases, the proposed system allowed effective discrimination (AUC > 0.9) between low and high levels of workload, vigilance, and stress even for high temporal resolution (<10 s). Finally, the generalizability of the proposed system has been tested through a cross-task calibration. The system calibrated with the data recorded during the laboratory tasks was able to discriminate the targeted human factors during the realistic task reaching AUC values higher than 0.8 at 40 s of temporal resolution in case of vigilance and workload, and 20 s of temporal resolution for the stress monitoring. These results pave the way for ecologic use of the system, where calibration data of the realistic task are difficult to obtain.

Dual Passive Reactive Brain-Computer Interface: A Novel Approach to Human-Machine Symbiosis.

The present study proposes a novel concept of neuroadaptive technology, namely a dual passive-reactive Brain-Computer Interface (BCI), that enables bi-directional interaction between humans and machines. We have implemented such a system in a realistic flight simulator using the NextMind classification algorithms and framework to decode pilots' intention (reactive BCI) and to infer their level of attention (passive BCI). Twelve pilots used the reactive BCI to perform checklists along with an anti-collision radar monitoring task that was supervised by the passive BCI. The latter simulated an automatic avoidance maneuver when it detected that pilots missed an incoming collision. The reactive BCI reached 100% classification accuracy with a mean reaction time of 1.6 s when exclusively performing the checklist task. Accuracy was up to 98.5% with a mean reaction time of 2.5 s when pilots also had to fly the aircraft and monitor the anti-collision radar. The passive BCI achieved a F1-score of 0.94. This first demonstration shows the potential of a dual BCI to improve human-machine teaming which could be applied to a variety of applications.

Investigating the Single Trial Detectability of Cognitive Face Processing by a Passive Brain-Computer Interface.

An automated recognition of faces enables machines to visually identify a person and to gain access to non-verbal communication, including mimicry. Different approaches in lab settings or controlled realistic environments provided evidence that automated face detection and recognition can work in principle, although applications in complex real-world scenarios pose a different kind of problem that could not be solved yet. Specifically, in autonomous driving-it would be beneficial if the car could identify non-verbal communication of pedestrians or other drivers, as it is a common way of communication in daily traffic. Automated identification from observation whether pedestrians or other drivers communicate through subtle cues in mimicry is an unsolved problem so far, as intent and other cognitive factors are hard to derive from observation. In contrast, communicating persons usually have clear understanding whether they communicate or not, and such information is represented in their mindsets. This work investigates whether the mental processing of faces can be identified through means of a Passive Brain-Computer Interface (pBCI). This then could be used to support the cars' autonomous interpretation of facial mimicry of pedestrians to identify non-verbal communication. Furthermore, the attentive driver can be utilized as a sensor to improve the context awareness of the car in partly automated driving. This work presents a laboratory study in which a pBCI is calibrated to detect responses of the fusiform gyrus in the electroencephalogram (EEG), reflecting face recognition. Participants were shown pictures from three different categories: faces, abstracts, and houses evoking different responses used to calibrate the pBCI. The resulting classifier could distinguish responses to faces from that evoked by other stimuli with accuracy above 70%, in a single trial. Further analysis of the classification approach and the underlying data identified activation patterns in the EEG that corresponds to face recognition in the fusiform gyrus. The resulting pBCI approach is promising as it shows better-than-random accuracy and is based on relevant and intended brain responses. Future research has to investigate whether it can be transferred from the laboratory to the real world and how it can be implemented into artificial intelligences, as used in autonomous driving.