Brain-computer Interface Communication Research Articles

Insula and inferior frontal triangularis activations distinguish between conditioned brain responses using emotional sounds for basic BCI communication.

In order to enable communication through a brain-computer interface (BCI), it is necessary to discriminate between distinct brain responses. As a first step, we probed the possibility to discriminate between affirmative (“yes”) and negative (“no”) responses using a semantic classical conditioning paradigm, within an fMRI setting. Subjects were presented with congruent and incongruent word-pairs as conditioned stimuli (CS), respectively eliciting affirmative and negative responses. Incongruent word-pairs were associated to an unpleasant unconditioned stimulus (scream, US1) and congruent word-pairs were associated to a pleasant unconditioned stimulus (baby-laughter, US2), in order to elicit emotional conditioned responses (CR). The aim was to discriminate between affirmative and negative responses, enabled by their association with the positive and negative affective stimuli. In the late acquisition phase, when the US were not present anymore, there was a strong significant differential activation for incongruent and congruent word-pairs in a cluster comprising the left insula and the inferior frontal triangularis. This association was not found in the habituation phase. These results suggest that the difference in affirmative and negative brain responses was established as an effect of conditioning, allowing to further investigate the possibility of using this paradigm for a binary choice BCI.

Multimodal brain-computer interface communication in disorders of consciousness

Brain-computer interfaces (BCIs) can provide a means of communication for people with severe motor impairments [1]. For people with traumatic or other brain injuries establishing communication has proven to be more problematic [2]. The main challenge is integrating a mechanism to deal with co-occurring disorders of consciousness. This abstract describes multimodal (auditory, tactile, visual) P300 BCI communication attempts performed with a 42-year-old male who had an intracranial hemorrhage of the cerebellum and brainstem one year prior to the measurements. The participant is paralyzed except for residual eye blinking and movement as well as some movement of the left hand. This residual muscular control is used for communication (termed conventional). Many pitfalls were encountered when attempting to establish communication. The main aspect is that when we performed the same task at different time points the results varied considerably (see Figure ​Figure11). Figure 1 Attempts performed to establish communication with the locked-in participant. Day zero was performed as an initial testing two months before the other measurements. The height of the bars indicates the offline classification rates (BCI with linear discriminant ...

Brain-computer interface (BCI) evaluation in people with amyotrophic lateral sclerosis

Brain-computer interfaces (BCIs) might restore communication to people severely disabled by amyotrophic lateral sclerosis (ALS) or other disorders. We sought to: 1) define a protocol for determining whether a person with ALS can use a visual P300-based BCI; 2) determine what proportion of this population can use the BCI; and 3) identify factors affecting BCI performance. Twenty-five individuals with ALS completed an evaluation protocol using a standard 6 × 6 matrix and parameters selected by stepwise linear discrimination. With an 8-channel EEG montage, the subjects fell into two groups in BCI accuracy (chance accuracy 3%). Seventeen averaged 92 (± 3)% (range 71–100%), which is adequate for communication (G70 group). Eight averaged 12 (± 6)% (range 0–36%), inadequate for communication (L40 subject group). Performance did not correlate with disability: 11/17 (65%) of G70 subjects were severely disabled (i.e. ALSFRS-R < 5). All L40 subjects had visual impairments (e.g. nystagmus, diplopia, ptosis). P300 was larger and more anterior in G70 subjects. A 16-channel montage did not significantly improve accuracy. In conclusion, most people severely disabled by ALS could use a visual P300-based BCI for communication. In those who could not, visual impairment was the principal obstacle. For these individuals, auditory P300-based BCIs might be effective.

Integrating language information with a hidden Markov model to improve communication rate in the P300 speller.

The P300 speller is a common brain-computer interface (BCI) application designed to communicate language by detecting event related potentials in a subject's electroencephalogram (EEG) signal. Information about the structure of natural language can be valuable for BCI communication systems, but few attempts have been made to incorporate this domain knowledge into the classifier. In this study, we treat BCI communication as a hidden Markov model (HMM) where hidden states are target characters and the EEG signal is the visible output. Using the Viterbi algorithm, language information can be incorporated in classification and errors can be corrected automatically. This method was first evaluated offline on a dataset of 15 healthy subjects who had a significant increase in bit rate from a previously published naïve Bayes approach and an average 32% increase from standard classification with dynamic stopping. An online pilot study of five healthy subjects verified these results as the average bit rate achieved using the HMM method was significantly higher than that using the naïve Bayes and standard methods. These findings strongly support the integration of domain-specific knowledge into BCI classification to improve system performance and accuracy.

Combining Motor Imagery With Selective Sensation Toward a Hybrid-Modality BCI

A hybrid modality brain-computer interface (BCI) is proposed in this paper, which combines motor imagery with selective sensation to enhance the discrimination between left and right mental tasks, e.g., the classification between left/ right stimulation sensation and right/ left motor imagery. In this paradigm, wearable vibrotactile rings are used to stimulate both the skin on both wrists. Subjects are required to perform the mental tasks according to the randomly presented cues (i.e., left hand motor imagery, right hand motor imagery, left stimulation sensation or right stimulation sensation). Two-way ANOVA statistical analysis showed a significant group effect (F (2,20) = 7.17, p = 0.0045), and the Benferroni-corrected multiple comparison test (with α = 0.05) showed that the hybrid modality group is 11.13% higher on average than the motor imagery group, and 10.45% higher than the selective sensation group. The hybrid modality experiment exhibits potentially wider spread usage within ten subjects crossed 70% accuracy, followed by four subjects in motor imagery and five subjects in selective sensation. Six subjects showed statistically significant improvement ( Benferroni-corrected) in hybrid modality in comparison with both motor imagery and selective sensation. Furthermore, among subjects having difficulties in both motor imagery and selective sensation, the hybrid modality improves their performance to 90% accuracy. The proposed hybrid modality BCI has demonstrated clear benefits for those poorly performing BCI users. Not only does the requirement of motor and sensory anticipation in this hybrid modality provide basic function of BCI for communication and control, it also has the potential for enhancing the rehabilitation during motor recovery.

Evaluating True BCI Communication Rate through Mutual Information and Language Models

Brain-computer interface (BCI) systems are a promising means for restoring communication to patients suffering from “locked-in” syndrome. Research to improve system performance primarily focuses on means to overcome the low signal to noise ratio of electroencephalogric (EEG) recordings. However, the literature and methods are difficult to compare due to the array of evaluation metrics and assumptions underlying them, including that: 1) all characters are equally probable, 2) character selection is memoryless, and 3) errors occur completely at random. The standardization of evaluation metrics that more accurately reflect the amount of information contained in BCI language output is critical to make progress. We present a mutual information-based metric that incorporates prior information and a model of systematic errors. The parameters of a system used in one study were re-optimized, showing that the metric used in optimization significantly affects the parameter values chosen and the resulting system performance. The results of 11 BCI communication studies were then evaluated using different metrics, including those previously used in BCI literature and the newly advocated metric. Six studies' results varied based on the metric used for evaluation and the proposed metric produced results that differed from those originally published in two of the studies. Standardizing metrics to accurately reflect the rate of information transmission is critical to properly evaluate and compare BCI communication systems and advance the field in an unbiased manner.

Brain communication in the locked-in state

Patients in the completely locked-in state have no means of communication and they represent the target population for brain-computer interface research in the last 15 years. Although different paradigms have been tested and different physiological signals used, to date no sufficiently documented completely locked-in state patient was able to control a brain-computer interface over an extended time period. We introduce Pavlovian semantic conditioning to enable basic communication in completely locked-in state. This novel paradigm is based on semantic conditioning for online classification of neuroelectric or any other physiological signals to discriminate between covert (cognitive) 'yes' and 'no' responses. The paradigm comprised the presentation of affirmative and negative statements used as conditioned stimuli, while the unconditioned stimulus consisted of electrical stimulation of the skin paired with affirmative statements. Three patients with advanced amyotrophic lateral sclerosis participated over an extended time period, one of which was in a completely locked-in state, the other two in the locked-in state. The patients' level of vigilance was assessed through auditory oddball procedures to study the correlation between vigilance level and the classifier's performance. The average online classification accuracies of slow cortical components of electroencephalographic signals were around chance level for all the patients. The use of a non-linear classifier in the offline classification procedure resulted in a substantial improvement of the accuracy in one locked-in state patient achieving 70% correct classification. A reliable level of performance in the completely locked-in state patient was not achieved uniformly throughout the 37 sessions despite intact cognitive processing capacity, but in some sessions communication accuracies up to 70% were achieved. Paradigm modifications are proposed. Rapid drop of vigilance was detected suggesting attentional variations or variations of circadian period as important factors in brain-computer interface communication with locked-in state and completely locked-in state.

Amyotrophic lateral sclerosis progression and stability of brain-computer interface communication

Our objective was to investigate the relationship between brain-computer interface (BCI) communication skill and disease progression in amyotrophic lateral sclerosis (ALS). We sought also to assess stability of BCI communication performance over time and whether it is related to the progression of neurological impairment before entering the locked-in state. A three years follow-up, BCI evaluation in a group of ALS patients (n = 24) was conducted. For a variety of reasons only three patients completed the three years follow-up. BCI communication skill and disability level, using the Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised, were assessed at admission and at each of the three follow-ups. Multiple non-parametric statistical methods were used to ensure reliability of the dependent variables: correlations, paired test and factor analysis of variance. Results demonstrated no significant relationship between BCI communication skill (BCI-CS) and disease evolution. The patients who performed the follow-up evaluations preserved their BCI-CS over time. Patients’ age at admission correlated positively with the ability to achieve control over a BCI. In conclusion, disease evolution in ALS does not affect the ability to control a BCI for communication. BCI performance can be maintained in the different stages of the illness.

Introduction of a Universal P300 Brain-Computer Interface Communication System

Article Introduction of a Universal P300 Brain-Computer Interface Communication System was published on September 7, 2013 in the journal Biomedical Engineering / Biomedizinische Technik (volume 58, issue SI-1-Track-S).

Semantic Classical Conditioning and Brain-Computer Interface Control: Encoding of Affirmative and Negative Thinking

The aim of the study was to investigate conditioned electroencephalography (EEG) responses to factually correct and incorrect statements in order to enable binary communication by means of a brain-computer interface (BCI). In two experiments with healthy participants true and false statements (serving as conditioned stimuli, CSs) were paired with two different tones which served as unconditioned stimuli (USs). The features of the USs were varied and tested for their effectiveness to elicit differentiable conditioned reactions (CRs). After acquisition of the CRs, these CRs to true and false statements were classified offline using a radial basis function kernel support vector machine. A mean single-trial classification accuracy of 50.5% was achieved for differentiating conditioned “yes” versus “no” thinking and mean accuracies of 65.4% for classification of “yes” and 68.8% for “no” thinking (both relative to baseline) were found using the best US. Analysis of the area under the curve of the conditioned EEG responses revealed significant differences between conditioned “yes” and “no” answers. Even though improvements are necessary, these first results indicate that the semantic conditioning paradigm could be a useful basis for further research regarding BCI communication in patients in the complete locked-in state.

Automatic single-trial discrimination of mental arithmetic, mental singing and the no-control state from prefrontal activity: toward a three-state NIRS-BCI

BackgroundNear-infrared spectroscopy (NIRS) is an optical imaging technology that has recently been investigated for use in a safe, non-invasive brain-computer interface (BCI) for individuals with severe motor impairments. To date, most NIRS-BCI studies have attempted to discriminate two mental states (e.g., a mental task and rest), which could potentially lead to a two-choice BCI system. In this study, we attempted to automatically differentiate three mental states - specifically, intentional activity due to 1) a mental arithmetic (MA) task and 2) a mental singing (MS) task, and 3) an unconstrained, "no-control (NC)" state - to investigate the feasibility of a three-choice system-paced NIRS-BCI.ResultsDeploying a dual-wavelength frequency domain near-infrared spectrometer, we interrogated nine sites around the frontopolar locations while 7 able-bodied adults performed mental arithmetic and mental singing to answer multiple-choice questions within a system-paced paradigm. With a linear classifier trained on a ten-dimensional feature set, an overall classification accuracy of 56.2% was achieved for the MA vs. MS vs. NC classification problem and all individual participant accuracies significantly exceeded chance (i.e., 33%). However, as anticipated based on results of previous work, the three-class discrimination was unsuccessful for three participants due to the ineffectiveness of the mental singing task. Excluding these three participants increases the accuracy rate to 62.5%. Even without training, three of the remaining four participants achieved accuracies approaching 70%, the value often cited as being necessary for effective BCI communication.ConclusionsThese results are encouraging and demonstrate the potential of a three-state system-paced NIRS-BCI with two intentional control states corresponding to mental arithmetic and mental singing.

An online brain–computer interface based on shifting attention to concurrent streams of auditory stimuli

We report on the development and online testing of an electroencephalogram-based brain–computer interface (BCI) that aims to be usable by completely paralysed users—for whom visual or motor-system-based BCIs may not be suitable, and among whom reports of successful BCI use have so far been very rare. The current approach exploits covert shifts of attention to auditory stimuli in a dichotic-listening stimulus design. To compare the efficacy of event-related potentials (ERPs) and steady-state auditory evoked potentials (SSAEPs), the stimuli were designed such that they elicited both ERPs and SSAEPs simultaneously. Trial-by-trial feedback was provided online, based on subjects' modulation of N1 and P3 ERP components measured during single 5 s stimulation intervals. All 13 healthy subjects were able to use the BCI, with performance in a binary left/right choice task ranging from 75% to 96% correct across subjects (mean 85%). BCI classification was based on the contrast between stimuli in the attended stream and stimuli in the unattended stream, making use of every stimulus, rather than contrasting frequent standard and rare ‘oddball’ stimuli. SSAEPs were assessed offline: for all subjects, spectral components at the two exactly known modulation frequencies allowed discrimination of pre-stimulus from stimulus intervals, and of left-only stimuli from right-only stimuli when one side of the dichotic stimulus pair was muted. However, attention modulation of SSAEPs was not sufficient for single-trial BCI communication, even when the subject's attention was clearly focused well enough to allow classification of the same trials via ERPs. ERPs clearly provided a superior basis for BCI. The ERP results are a promising step towards the development of a simple-to-use, reliable yes/no communication system for users in the most severely paralysed states, as well as potential attention-monitoring and -training applications outside the context of assistive technology.

Online use of error-related potentials in healthy users and people with severe motor impairment increases performance of a P300-BCI

ObjectiveTo investigate whether error-related potentials can be used to increase information transfer rate of a P3 brain–computer interface (BCI) in healthy and motor-impaired individuals. MethodsExtraction and classification of the error-related potential was performed offline on data recorded from six amyotrophic lateral sclerosis (ALS) patients. An online study with 17 healthy and six motor impaired participants followed, using a modified P3 speller to provide explicit feedback of spelled letters. On recognition of error-related potentials, the interface informed users that the incorrect letter was automatically deleted. ResultsThe offline cross-validation estimate of P3 speller data of six ALS patients increased bit rate by 0.44bit/trial. During online copy spelling, the participants increased their bit rate by 0.52bit/trial with the error correction system (ECS). Some participants performed free spelling and were able to increase their bit rate. Finally, we demonstrated that healthy participants could increase their bit rate by using a classifier pre-trained on other users’ data. ConclusionsError-related potentials as a secondary source of information can be used to increase overall bit rate in a P3 BCI. SignificanceThe method should be made available to any patient using the P3 BCI for communication.

Classical conditioning of the BOLD signal as a paradigm for basic BCI communication in Alzheimer patients

Background Alzheimer patients express their need of social interaction even when their communication abilities are highly impaired. (Mayhew et al., 2001). Brain-computer interfaces (BCI), already used with severely paralyzed patients, may be adapted for communicating with Alzheimer patients by shifting the paradigm from instrumental-operant learning to classical conditioning (Birbaumer, 2006), e.g. by associating “yes” thinking to a positive emotion and “no” thinking to a negative emotion. We designed an fMRI-based BCI setting aimed at conditioning subjects to associate positive and negative emotional stimuli with respectively congruent and incongruent word pairs, in view of a BCI application for basic yes/no communication. Methods fMRI was performed on 6 healthy subjects during a classical conditioning session, comprising the phases of habituation, acquisition and extinction. The unconditioned stimuli consisted of a positive (baby laughter) and negative (scream) emotional sound. The conditioned stimuli, presented aurally, were congruent (e.g. “animal-elephant”) and incongruent (e.g. “animal-Germany”) word pairs. During the conditioning acquisition phase, congruent word pairs were associated to the baby laughter and incongruent word pairs were associated to the scream. A linear Support Vector Machine was implemented to classify the BOLD signal corresponding to congruent and incongruent word pairs. To investigate the relative importance of distinct brain areas in decoding different brain states, feature vectors from the frontal cortex were also used as input to build a separate classifier. Results Participants rated in the Self Assessment Manikin (SAM) the scream and the laughter as having, respectively, negative and positive valence. Moreover, the scream was associated to higher arousal compared to the baby laughter. Classification of the BOLD signal as a response to the congruent and incongruent word pairs immediately followed by the emotional unconditioned stimuli showed above chance level performance (57-64%) in one subject, while the performance was chance level on the remaining subjects (44-56%). Conclusions We presented a possible paradigm for basic yes/no communication, based on the conditioning of BOLD signal by the repeated association of emotional and semantic stimuli. The performance of the classifier by feature selection needs to be improved before being tested with Alzheimer patients. Future developments also comprise an online implementation of the system.

A Brain–Computer Interface Controlled Auditory Event‐Related Potential (P300) Spelling System for Locked‐In Patients

Using brain-computer interfaces (BCI) humans can select letters or other targets on a computer screen without any muscular involvement. An intensively investigated kind of BCI is based on the recording of visual event-related brain potentials (ERP). However, some severely paralyzed patients who need a BCI for communication have impaired vision or lack control of gaze movement, thus making a BCI depending on visual input no longer feasible. In an effort to render the ERP-BCI usable for this group of patients, the ERP-BCI was adapted to auditory stimulation. Letters of the alphabet were assigned to cells in a 5 x 5 matrix. Rows of the matrix were coded with numbers 1 to 5, and columns with numbers 6 to 10, and the numbers were presented auditorily. To select a letter, users had to first select the row and then the column containing the desired letter. Four severely paralyzed patients in the end-stage of a neurodegenerative disease were examined. All patients performed above chance level. Spelling accuracy was significantly lower with the auditory system as compared with a similar visual system. Patients reported difficulties in concentrating on the task when presented with the auditory system. In future studies, the auditory ERP-BCI should be adjusted by taking into consideration specific features of severely paralyzed patients, such as reduced attention span. This adjustment in combination with more intensive training will show whether an auditory ERP-BCI can become an option for visually impaired patients.

An enhancement of EEG-based brain-computer interface communication software

An enhancement of EEG-based brain-computer interface communication software

P300-Based Brain-Computer Interface Communication: Evaluation and Follow-up in Amyotrophic Lateral Sclerosis.

To describe results of training and 1-year follow-up of brain-communication in a larger group of early and middle stage amyotrophic lateral sclerosis (ALS) patients using a P300-based brain–computer interface (BCI), and to investigate the relationship between clinical status, age and BCI performance. A group of 21 ALS patients were tested with a BCI-system using two-dimensional cursor movements. A four choice visual paradigm was employed to training and test the brain-communication abilities. The task consisted of reaching with the cursor one out of four icons representing four basic needs. Five patients performed a follow-up test 1 year later. The clinical severity in all patients were assessed with a battery of clinical tests. A comparable control group of nine healthy subjects was employed to investigate performance differences. Nineteen patients and nine healthy subjects were able to achieve good and excellent cursor movements' control, acquiring at least communication abilities above chance level; during follow-up the patients maintained their BCI-skill. We found mild cognitive impairments in the ALS group which may be attributed to motor deficiencies, while no relevant correlation has been found between clinical data and BCI performance. A positive correlation between age and the BCI-skill in patients was found. Time since training acquisition and clinical status did not affect the patients brain-communication skill at early and middle stage of the disease. A brain-communication tool can be used in most ALS patients at early and middle stage of the disease before entering the locked-in stage.

An MEG-based brain–computer interface (BCI)

Brain–computer interfaces (BCIs) allow for communicating intentions by mere brain activity, not involving muscles. Thus, BCIs may offer patients who have lost all voluntary muscle control the only possible way to communicate. Many recent studies have demonstrated that BCIs based on electroencephalography (EEG) can allow healthy and severely paralyzed individuals to communicate. While this approach is safe and inexpensive, communication is slow. Magnetoencephalography (MEG) provides signals with higher spatiotemporal resolution than EEG and could thus be used to explore whether these improved signal properties translate into increased BCI communication speed. In this study, we investigated the utility of an MEG-based BCI that uses voluntary amplitude modulation of sensorimotor μ and β rhythms. To increase the signal-to-noise ratio, we present a simple spatial filtering method that takes the geometric properties of signal propagation in MEG into account, and we present methods that can process artifacts specifically encountered in an MEG-based BCI. Exemplarily, six participants were successfully trained to communicate binary decisions by imagery of limb movements using a feedback paradigm. Participants achieved significant μ rhythm self control within 32 min of feedback training. For a subgroup of three participants, we localized the origin of the amplitude modulated signal to the motor cortex. Our results suggest that an MEG-based BCI is feasible and efficient in terms of user training.

Brain-computer interfaces: communication and restoration of movement in paralysis.

The review describes the status of brain-computer or brain-machine interface research. We focus on non-invasive brain-computer interfaces (BCIs) and their clinical utility for direct brain communication in paralysis and motor restoration in stroke. A large gap between the promises of invasive animal and human BCI preparations and the clinical reality characterizes the literature: while intact monkeys learn to execute more or less complex upper limb movements with spike patterns from motor brain regions alone without concomitant peripheral motor activity usually after extensive training, clinical applications in human diseases such as amyotrophic lateral sclerosis and paralysis from stroke or spinal cord lesions show only limited success, with the exception of verbal communication in paralysed and locked-in patients. BCIs based on electroencephalographic potentials or oscillations are ready to undergo large clinical studies and commercial production as an adjunct or a major assisted communication device for paralysed and locked-in patients. However, attempts to train completely locked-in patients with BCI communication after entering the complete locked-in state with no remaining eye movement failed. We propose that a lack of contingencies between goal directed thoughts and intentions may be at the heart of this problem. Experiments with chronically curarized rats support our hypothesis; operant conditioning and voluntary control of autonomic physiological functions turned out to be impossible in this preparation. In addition to assisted communication, BCIs consisting of operant learning of EEG slow cortical potentials and sensorimotor rhythm were demonstrated to be successful in drug resistant focal epilepsy and attention deficit disorder. First studies of non-invasive BCIs using sensorimotor rhythm of the EEG and MEG in restoration of paralysed hand movements in chronic stroke and single cases of high spinal cord lesions show some promise, but need extensive evaluation in well-controlled experiments. Invasive BMIs based on neuronal spike patterns, local field potentials or electrocorticogram may constitute the strategy of choice in severe cases of stroke and spinal cord paralysis. Future directions of BCI research should include the regulation of brain metabolism and blood flow and electrical and magnetic stimulation of the human brain (invasive and non-invasive). A series of studies using BOLD response regulation with functional magnetic resonance imaging (fMRI) and near infrared spectroscopy demonstrated a tight correlation between voluntary changes in brain metabolism and behaviour.

Brain-computer interface research at the Wadsworth Center.

Studies at the Wadsworth Center over the past 14 years have shown that people with or without motor disabilities can learn to control the amplitude of mu or beta rhythms in electroencephalographic (EEG) activity recorded from the scalp over sensorimotor cortex and can use that control to move a cursor on a computer screen in one or two dimensions. This EEG-based brain-computer interface (BCI) could provide a new augmentative communication technology for those who are totally paralyzed or have other severe motor impairments. Present research focuses on improving the speed and accuracy of BCI communication.