Brain-computer Interface Communication Research Articles

Enhancing photoacoustic imaging for lung diagnostics and BCI communication: simulation of cavity structures artifact generation and evaluation of noise reduction techniques

Pandemics like COVID-19 have highlighted the potential of Photoacoustic imaging (PAI) for Brain-Computer Interface (BCI) communication and lung diagnostics. However, PAI struggles with the clear imaging of blood vessels in areas like the lungs and brain due to their cavity structures. This paper presents a simulation model to analyze the generation and propagation mechanism within phantom tissues of PAI artifacts, focusing on the evaluation of both Anisotropic diffusion filtering (ADF) and Non-local mean (NLM) filtering, which significantly reduce noise and eliminate artifacts and signify a pivotal point for selecting artifact-removal algorithms under varying conditions of light distribution. Experimental validation demonstrated the efficacy of our technique, elucidating the effect of light source uniformity on artifact-removal performance. The NLM filtering simulation and ADF experimental validation increased the peak signal-to-noise ratio by 11.33% and 18.1%, respectively. The proposed technique adds a promising dimension for BCI and is an accurate imaging solution for diagnosing lung diseases.

Online speech synthesis using a chronically implanted brain–computer interface in an individual with ALS

Brain–computer interfaces (BCIs) that reconstruct and synthesize speech using brain activity recorded with intracranial electrodes may pave the way toward novel communication interfaces for people who have lost their ability to speak, or who are at high risk of losing this ability, due to neurological disorders. Here, we report online synthesis of intelligible words using a chronically implanted brain-computer interface (BCI) in a man with impaired articulation due to ALS, participating in a clinical trial (ClinicalTrials.gov, NCT03567213) exploring different strategies for BCI communication. The 3-stage approach reported here relies on recurrent neural networks to identify, decode and synthesize speech from electrocorticographic (ECoG) signals acquired across motor, premotor and somatosensory cortices. We demonstrate a reliable BCI that synthesizes commands freely chosen and spoken by the participant from a vocabulary of 6 keywords previously used for decoding commands to control a communication board. Evaluation of the intelligibility of the synthesized speech indicates that 80% of the words can be correctly recognized by human listeners. Our results show that a speech-impaired individual with ALS can use a chronically implanted BCI to reliably produce synthesized words while preserving the participant’s voice profile, and provide further evidence for the stability of ECoG for speech-based BCIs.

Brain-Computer Interfaces for Communication in Patients with Disorders of Consciousness: A Gap Analysis and Scientific Roadmap.

We developed a gap analysis that examines the role of brain-computer interfaces (BCI) in patients with disorders of consciousness (DoC), focusing on their assessment, establishment of communication, and engagement with their environment. The Curing Coma Campaign convened a Coma Science work group that included 16 clinicians and neuroscientists with expertise in DoC. The work group met online biweekly and performed a gap analysis of the primary question. We outline a roadmap for assessing BCI readiness in patients with DoC and for advancing the use of BCI devices in patients with DoC. Additionally, we discuss preliminary studies that inform development of BCI solutions for communication and assessment of readiness for use of BCIs in DoC study participants. Special emphasis is placed on the challenges posed by the complex pathophysiologies caused by heterogeneous brain injuries and their impact on neuronal signaling. The differences between one-way and two-way communication are specifically considered. Possible implanted and noninvasive BCI solutions for acute and chronic DoC in adult and pediatric populations are also addressed. We identify clinical and technical gaps hindering the use of BCI in patients with DoC in each of these contexts and provide a roadmap for research aimed at improving communication for adults and children with DoC, spanning the clinical spectrum from intensive care unit to chronic care.

Towards Implantable Brain-Computer Interface for Communication in Locked-In Syndrome patients

Abstract Locked-in Syndrome (LIS) severely restricts the communication abilities of individuals due to extensive paralysis. The Intracranial Neuro Telemetry to Restore communication (INTRECOM) project aims to aid patients in overcoming these limitations by developing a fully implantable brain computer interface (BCI) system based on state-of-the-art technology holding great promise in revolutionizing the lives of LIS patients. In this project, the Graz BCI group, with its expertise in understanding brain dynamics associated with movement, focuses on the algorithmic development of advanced decoders that enable the user to execute specific commands by simply attempting corresponding movements. Our preliminary findings using electrocorticography (ECoG) data obtained from one individual with LIS that underwent implantation of a BCI communication system demonstrate the applicability of a ‘brain switch’ function that detects brain signals associated with attempted movements. This switch can then be used to translate the intention of the user into a click/select function on a screen.

It takes two (seconds): decreasing encoding time for two-choice functional near-infrared spectroscopy brain-computer interface communication.

Brain-computer interfaces (BCIs) can provide severely motor-impaired patients with a motor-independent communication channel. Functional near-infrared spectroscopy (fNIRS) constitutes a promising BCI-input modality given its high mobility, safety, user comfort, cost-efficiency, and relatively low motion sensitivity. The present study aimed at developing an efficient and convenient two-choice fNIRS communication BCI by implementing a relatively short encoding time (2s), considerably increasing communication speed, and decreasing the cognitive load of BCI users. To encode binary answers to 10 biographical questions, 10 healthy adults repeatedly performed a combined motor-speech imagery task within 2 different time windows guided by auditory instructions. Each answer-encoding run consisted of 10 trials. Answers were decoded during the ongoing experiment from the time course of the individually identified most-informative fNIRS channel-by-chromophore combination. The answers of participants were decoded online with an accuracy of 85.8% (run-based group mean). Post-hoc analysis yielded an average single-trial accuracy of 68.1%. Analysis of the effect of number of trial repetitions showed that the best information-transfer rate could be obtained by combining four encoding trials. The study demonstrates that an encoding time as short as 2s can enable immediate, efficient, and convenient fNIRS-BCI communication.

Talking with hands and feet: Selective somatosensory attention and fMRI enable robust and convenient brain-based communication

In brain-based communication, voluntarily modulated brain signals (instead of motor output) are utilized to interact with the outside world. The possibility to circumvent the motor system constitutes an important alternative option for severely paralyzed. Most communication brain-computer interface (BCI) paradigms require intact visual capabilities and impose a high cognitive load, but for some patients, these requirements are not given. In these situations, a better-suited, less cognitively demanding information-encoding approach may exploit auditorily-cued selective somatosensory attention to vibrotactile stimulation. Here, we propose, validate and optimize a novel communication-BCI paradigm using differential fMRI activation patterns evoked by selective somatosensory attention to tactile stimulation of the right hand or left foot. Using cytoarchitectonic probability maps and multi-voxel pattern analysis (MVPA), we show that the locus of selective somatosensory attention can be decoded from fMRI-signal patterns in (especially primary) somatosensory cortex with high accuracy and reliability, with the highest classification accuracy (85.93%) achieved when using Brodmann area 2 (SI-BA2) at a probability level of 0.2. Based on this outcome, we developed and validated a novel somatosensory attention-based yes/no communication procedure and demonstrated its high effectiveness even when using only a limited amount of (MVPA) training data. For the BCI user, the paradigm is straightforward, eye-independent, and requires only limited cognitive functioning. In addition, it is BCI-operator friendly given its objective and expertise-independent procedure. For these reasons, our novel communication paradigm has high potential for clinical applications.

Novel electrotactile brain-computer interface with somatosensory event-related potential based control.

A brain computer interface (BCI) allows users to control external devices using non-invasive brain recordings, such as electroencephalography (EEG). We developed and tested a novel electrotactile BCI prototype based on somatosensory event-related potentials (sERP) as control signals, paired with a tactile attention task as a control paradigm. A novel electrotactile BCI comprises commercial EEG device, an electrical stimulator and custom software for EEG recordings, electrical stimulation control, synchronization between devices, signal processing, feature extraction, selection, and classification. We tested a novel BCI control paradigm based on tactile attention on a sensation at a target stimulation location on the forearm. Tactile stimuli were electrical pulses delivered at two proximal locations on the user's forearm for stimulating branches of radial and median nerves, with equal probability of the target and distractor stimuli occurrence, unlike in any other ERP-based BCI design. We proposed a compact electrical stimulation electrodes configuration for delivering electrotactile stimuli (target and distractor) using 2 stimulation channels and 3 stimulation electrodes. We tested the feasibility of a single EEG channel BCI control, to determine pseudo-online BCI performance, in ten healthy subjects. For optimizing the BCI performance we compared the results for two classifiers, sERP averaging approaches, and novel dedicated feature extraction/selection methods via cross-validation procedures. We achieved a single EEG channel BCI classification accuracy in the range of 75.1 to 88.1% for all subjects. We have established an optimal combination of: single trial averaging to obtain sERP, feature extraction/selection methods and classification approach. The obtained results demonstrate that a novel electrotactile BCI paradigm with equal probability of attended (target) and unattended (distractor) stimuli and proximal stimulation sites is feasible. This method may be used to drive restorative BCIs for sensory retraining in stroke or brain injury, or assistive BCIs for communication in severely disabled users.

Ethical issues raised by incorporating personalized language models into brain-computer interface communication technologies: a qualitative study of individuals with neurological disease

Purpose To examine the views of individuals with neurodegenerative diseases about ethical issues related to incorporating personalized language models into brain-computer interface (BCI) communication technologies. Methods Fifteen semi-structured interviews and 51 online free response surveys were completed with individuals diagnosed with neurodegenerative disease that could lead to loss of speech and motor skills. Each participant responded to questions after six hypothetical ethics vignettes were presented that address the possibility of building language models with personal words and phrases in BCI communication technologies. Data were analyzed with consensus coding, using modified grounded theory. Results Four themes were identified. (1) The experience of a neurodegenerative disease shapes preferences for personalized language models. (2) An individual’s identity will be affected by the ability to personalize the language model. (3) The motivation for personalization is tied to how relationships can be helped or harmed. (4) Privacy is important to people who may need BCI communication technologies. Responses suggest that the inclusion of personal lexica raises ethical issues. Stakeholders want their values to be considered during development of BCI communication technologies. Conclusions With the rapid development of BCI communication technologies, it is critical to incorporate feedback from individuals regarding their ethical concerns about the storage and use of personalized language models. Stakeholder values and preferences about disability, privacy, identity and relationships should drive design, innovation and implementation. IMPLICATIONS FOR REHABILITATION Individuals with neurodegenerative diseases are important stakeholders to consider in development of natural language processing within brain-computer interface (BCI) communication technologies. The incorporation of personalized language models raises issues related to disability, identity, relationships, and privacy. People who may one day rely on BCI communication technologies care not just about usability of communication technology but about technology that supports their values and priorities. Qualitative ethics-focused research is a valuable tool for exploring stakeholder perspectives on new capabilities of BCI communication technologies, such as the storage and use of personalized language models.

EEG-based covert speech decoding using random rotation extreme learning machine ensemble for intuitive BCI communication

Background and motivation:Brain–computer interfaces (BCI) assist communication for the handicapped and disabled. Previous work has shown that it is usually non-intuitive, and it frequently employs Electroencephalogram (EEG) based motor imagery. Ensemble learning has been shown to be reliable in a variety of BCI classification tasks, including motor imagery and P300 event-related potential. The purpose of ensemble learning, according to past research, is to get trained accurate but diverse base classifiers to increase generalization performance. Extreme Learning Machine (ELM) with fast learning speed has been proven effective in classification applications and suitable for ensemble learning due to its randomness. The suitability of ensemble methods for intuitive BCI communication using EEG data of different covert speech words is investigated in this work. Methods:To achieve this goal, we construct a random rotation based kernel ELM ensemble, resulting in very diverse classifiers which can learn through transformed spaces. Minimize error ensemble pruning is used to prune low complementarity classifiers, and consequently, combining the remaining high complementarity classifiers to get the final ensemble classifier. Eight subjects performed covert speech tasks which involved mentally repeating five words namely; ‘left’, ‘right’, ‘up’, ‘down’, and ‘stop’. Results:We achieved classification accuracy with pruning and without pruning of 63.67% and 57.13% respectively. Experimental results show that, in comparison to other classification techniques, the proposed technique produces competitive results. Conclusion and Significance:Our findings indicate the potential of random rotation kernel ELM ensemble pruning to effectively classify EEG-based covert speech signals. This proposed research involving an ensemble method for the classification of covert speech words can make a significant contribution towards intuitive BCI development using silent speech.

A pediatric near-infrared spectroscopy brain-computer interface based on the detection of emotional valence.

Brain-computer interfaces (BCIs) are being investigated as an access pathway to communication for individuals with physical disabilities, as the technology obviates the need for voluntary motor control. However, to date, minimal research has investigated the use of BCIs for children. Traditional BCI communication paradigms may be suboptimal given that children with physical disabilities may face delays in cognitive development and acquisition of literacy skills. Instead, in this study we explored emotional state as an alternative access pathway to communication. We developed a pediatric BCI to identify positive and negative emotional states from changes in hemodynamic activity of the prefrontal cortex (PFC). To train and test the BCI, 10 neurotypical children aged 8–14 underwent a series of emotion-induction trials over four experimental sessions (one offline, three online) while their brain activity was measured with functional near-infrared spectroscopy (fNIRS). Visual neurofeedback was used to assist participants in regulating their emotional states and modulating their hemodynamic activity in response to the affective stimuli. Child-specific linear discriminant classifiers were trained on cumulatively available data from previous sessions and adaptively updated throughout each session. Average online valence classification exceeded chance across participants by the last two online sessions (with 7 and 8 of the 10 participants performing better than chance, respectively, in Sessions 3 and 4). There was a small significant positive correlation with online BCI performance and age, suggesting older participants were more successful at regulating their emotional state and/or brain activity. Variability was seen across participants in regards to BCI performance, hemodynamic response, and discriminatory features and channels. Retrospective offline analyses yielded accuracies comparable to those reported in adult affective BCI studies using fNIRS. Affective fNIRS-BCIs appear to be feasible for school-aged children, but to further gauge the practical potential of this type of BCI, replication with more training sessions, larger sample sizes, and end-users with disabilities is necessary.

Intracranial brain-computer interface spelling using localized visual motion response

Intracranial brain-computer interfaces (BCIs) can assist severely disabled persons in text communication and environmental control with high precision and speed. Nevertheless, sustainable BCI implants require minimal invasiveness. One of the implantation strategies is to adopt localized and robust cortical activities to drive BCI communication and to make a precise presurgical planning. The visual motion response is a good candidate for inclusion in this strategy because of its focal activity over the middle temporal visual area (MT). Here, we developed an intracranial BCI for spelling, utilizing only three electrodes over the MT area. The best recording electrodes were decided by preoperative functional magnetic resonance imaging (MRI) localization of the MT, and local neural activities were further enhanced by differential rereferencing of these electrodes. The BCI spelling system was validated both offline and online by five epilepsy patients, achieving the fastest speed of 62 bits/min, i.e., 12 characters/min. Moreover, the response patterns of dual-directional visual motion stimuli provided an additional dimension of BCI target encoding and paved the way for a higher information transfer rate of intracranial BCI spelling.

Application of brain-computer-interface in awareness detection using machine learning methods

The awareness detection in patients with disorders of consciousness currently relies on behavioral observations and CRS-R tests, however, the mis-diagnosis rates have been relatively high. In this study, we applied brain-computer interface (BCI) to awareness detection with a passive auditory stimulation paradigm. 12 subjects with normal hearing were invited to collect electroencephalogram (EEG) based on a BCI communication system, in which EEG signals are transmitted wirelessly. After necessary preprocessing, RBF-SVM and EEGNet were used for algorithm realization and analysis. For a single subject, RBF-SVM can distinguish his (her) name stimuli awareness with classification accuracies ranging from 60–95%. EEGNet was used to learn all subjects' data and improved accuracy to 78.04% for characteristics finding and model generalization. Moreover, we completed the supplementary analysis work from the time domain and time-frequency domain. This study applied BCI communication to human awareness detection, proposed a passive auditory paradigm, and proved the effectiveness, which could be an inspiration for brain, mental or physical diseases diagnosis and detection.

Bioelectricity Buzz

Bioelectricity Buzz

Yes/No Classification of EEG data from CLIS patients.

The goal of this research is to evaluate the usability of new features to classify EEG data from several completely locked-in patients (CLIS), and eventually build a more reliable communication system for them. Patients in such state are completely paralyzed, preventing them to be able to talk, but they retain their cognitive abilities.The data were obtained from four CLIS patients and recorded during an auditory paradigm task during which they were asked yes/no questions. Spectral measures such as the relative power of δ, θ, α, β and γ frequency bands, spectral edge frequencies (SEF50 and SEF95), complexity measure obtained from Poincaré plots and connectivity measures such as the imaginary part of coherency and the weighted Symbolic Mutual Information (wSMI) were used as features. The data was classified using Random Forest and Support Vector Machine, two methods successfully used to classify mental states in both healthy subjects and patients. Additionally, two cases were studied. The first case uses data recorded when the patient is answering questions, while in the second case it also includes data recorded when the experimenter is asking the questions.The classification accuracy during training varies between 51.73 to 67.72% in the first case, and from 50.41 to 67.94% for the second case. Overall, wSMI with a time lag of 64 ms gave the best classification accuracy and in general, Random Forest appears to be the best classification method.Clinical relevance This case study investigates the usability of new features based on EEG complexity and connectivity to classify CLIS patients brain signal, what results in a further step toward the demand of more effective EEG-based Brain-Computer Interface communication systems for CLIS patients.

Interface, interaction, and intelligence in generalized brain–computer interfaces

A brain-computer interface (BCI) establishes a direct communication channel between a brain and an external device. With recent advances in neurotechnology and artificial intelligence (AI), the brain signals in BCI communication have been advanced from sensation and perception to higher-level cognition activities. While the field of BCI has grown rapidly in the past decades, the core technologies and innovative ideas behind seemingly unrelated BCI systems have never been summarized from an evolutionary point of view. Here, we review various BCI paradigms and present an evolutionary model of generalized BCI technology which comprises three stages: interface, interaction, and intelligence (I3). We also highlight challenges, opportunities, and future perspectives in the development of new BCI technology.

A functional BCI model by the P2731 working group: psychology

ABSTRACT The development of Brain-Computer Interfaces (BCI) gathers experts and specialists in various fields, such as engineering, computer science, medicine, or cognitive neuroscience. Each of these disciplines has specific terminology, which makes mutual understanding and research collaboration difficult. The IEEE P2731 working group aims to improve communication between BCI researchers by developing a functional model and standards for terminology that can be used as a common description framework for all the involved knowledge fields. This work focuses on the vocabulary of mental processes involved in BCI communication and describes their role in a Functional Model that considers their influence on BCI performance. Finally, it presents potential uses of the proposed model.

Subject-Independent Functional Near-Infrared Spectroscopy-Based Brain-Computer Interfaces Based on Convolutional Neural Networks.

Functional near-infrared spectroscopy (fNIRS) has attracted increasing attention in the field of brain–computer interfaces (BCIs) owing to their advantages such as non-invasiveness, user safety, affordability, and portability. However, fNIRS signals are highly subject-specific and have low test-retest reliability. Therefore, individual calibration sessions need to be employed before each use of fNIRS-based BCI to achieve a sufficiently high performance for practical BCI applications. In this study, we propose a novel deep convolutional neural network (CNN)-based approach for implementing a subject-independent fNIRS-based BCI. A total of 18 participants performed the fNIRS-based BCI experiments, where the main goal of the experiments was to distinguish a mental arithmetic task from an idle state task. Leave-one-subject-out cross-validation was employed to evaluate the average classification accuracy of the proposed subject-independent fNIRS-based BCI. As a result, the average classification accuracy of the proposed method was reported to be 71.20 ± 8.74%, which was higher than the threshold accuracy for effective BCI communication (70%) as well as that obtained using conventional shrinkage linear discriminant analysis (65.74 ± 7.68%). To achieve a classification accuracy comparable to that of the proposed subject-independent fNIRS-based BCI, 24 training trials (of approximately 12 min) were necessary for the traditional subject-dependent fNIRS-based BCI. It is expected that our CNN-based approach would reduce the necessity of long-term individual calibration sessions, thereby enhancing the practicality of fNIRS-based BCIs significantly.

Brain-Computer Interfaces for Communication: Preferences of Individuals With Locked-in Syndrome

Background Brain-computer interfaces (BCIs) have been proposed as an assistive technology (AT) allowing people with locked-in syndrome (LIS) to use neural signals to communicate. To design a communication BCI (cBCI) that is fully accepted by the users, their opinion should be taken into consideration during the research and development process. Objective We assessed the preferences of prospective cBCI users regarding (1) the applications they would like to control with a cBCI, (2) the mental strategies they would prefer to use to control the cBCI, and (3) when during their clinical trajectory they would like to be informed about AT and cBCIs. Furthermore, we investigated if individuals diagnosed with progressive and sudden onset (SO) disorders differ in their opinion. Methods We interviewed 28 Dutch individuals with LIS during a 3-hour home visit using multiple-choice, ranking, and open questions. During the interview, participants were informed about BCIs and the possible mental strategies. Results Participants rated (in)direct forms of communication, computer use, and environmental control as the most desired cBCI applications. In addition, active cBCI control strategies were preferred over reactive strategies. Furthermore, individuals with progressive and SO disorders preferred to be informed about AT and cBCIs at the moment they would need it. Conclusions We show that individuals diagnosed with progressive and SO disorders preferred, in general, the same applications, mental strategies, and time of information. By collecting the opinion of a large sample of individuals with LIS, this study provides valuable information to stakeholders in cBCI and other AT development.

Impact of Stimulus Features on the Performance of a Gaze-Independent Brain-Computer Interface Based on Covert Spatial Attention Shifts.

Regaining communication abilities in patients who are unable to speak or move is one of the main goals in decoding brain waves for brain-computer interface (BCI) control. Many BCI approaches designed for communication rely on attention to visual stimuli, commonly applying an oddball paradigm, and require both eye movements and adequate visual acuity. These abilities may, however, be absent in patients who depend on BCI communication. We have therefore developed a response-based communication BCI, which is independent of gaze shifts but utilizes covert shifts of attention to the left or right visual field. We recorded the electroencephalogram (EEG) from 29 channels and coregistered the vertical and horizontal electrooculogram. Data-driven decoding of small attention-based differences between the hemispheres, also known as N2pc, was performed using 14 posterior channels, which are expected to reflect correlates of visual spatial attention. Eighteen healthy participants responded to 120 statements by covertly directing attention to one of two colored symbols (green and red crosses for “yes” and “no,” respectively), presented in the user’s left and right visual field, respectively, while maintaining central gaze fixation. On average across participants, 88.5% (std: 7.8%) of responses were correctly decoded online. In order to investigate the potential influence of stimulus features on accuracy, we presented the symbols with different visual angles, by altering symbol size and eccentricity. The offline analysis revealed that stimulus features have a minimal impact on the controllability of the BCI. Hence, we show with our novel approach that spatial attention to a colored symbol is a robust method with which to control a BCI, which has the potential to support severely paralyzed people with impaired eye movements and low visual acuity in communicating with their environment.

Neural Decoding of Imagined Speech and Visual Imagery as Intuitive Paradigms for BCI Communication.

Brain-computer interface (BCI) is oriented toward intuitive systems that users can easily operate. Imagined speech and visual imagery are emerging paradigms that can directly convey a user's intention. We investigated the underlying characteristics that affect the decoding performance of these two paradigms. Twenty-two subjects performed imagined speech and visual imagery of twelve words/phrases frequently used for patients' communication. Spectral features were analyzed with thirteen-class classification (including rest class) using EEG filtered in six frequency ranges. In addition, cortical regions relevant to the two paradigms were analyzed by classification using single-channel and pre-defined cortical groups. Furthermore, we analyzed the word properties that affect the decoding performance based on the number of syllables, concrete and abstract concepts, and the correlation between the two paradigms. Finally, we investigated multiclass scalability in both paradigms. The high-frequency band displayed a significantly superior performance to that in the case of any other spectral features in the thirteen-class classification (imagined speech: 39.73 ± 5.64%; visual imagery: 40.14 ± 4.17%). Furthermore, the performance of Broca's and Wernicke's areas and auditory cortex was found to have improved among the cortical regions in both paradigms. As the number of classes increased, the decoding performance decreased moderately. Moreover, every subject exceeded the confidence level performance, implying the strength of the two paradigms in BCI inefficiency. These two intuitive paradigms were found to be highly effective for multiclass communication systems, having considerable similarities between each other. The results could provide crucial information for improving the decoding performance for practical BCI applications.