Specific Mental Tasks Research Articles

Improving classification performance of motor imagery BCI through EEG data augmentation with conditional generative adversarial networks

In brain-computer interface (BCI), building accurate electroencephalogram (EEG) classifiers for specific mental tasks is critical for BCI performance. The classifiers are developed by machine learning (ML) and deep learning (DL) techniques, requiring a large dataset for training to build reliable and accurate models. However, collecting large enough EEG datasets is difficult due to intra-/inter-subject variabilities and experimental costs. This leads to the data scarcity problem, which causes overfitting issues to training samples, resulting in reducing generalization performance. To solve the EEG data scarcity problem and improve the performance of the EEG classifiers, we propose a novel EEG data augmentation (DA) framework using conditional generative adversarial networks (cGANs). An experimental study is implemented with two public EEG datasets, including motor imagery (MI) tasks (BCI competition IV IIa and III IVa), to validate the effectiveness of the proposed EEG DA method for the EEG classifiers. To evaluate the proposed cGAN-based DA method, we tested eight EEG classifiers for the experiment, including traditional MLs and state-of-the-art DLs with three existing EEG DA methods. Experimental results showed that most DA methods with proper DA proportion in the training dataset had higher classification performances than without DA. Moreover, applying the proposed DA method showed superior classification performance improvement than the other DA methods. This shows that the proposed method is a promising EEG DA method for enhancing the performances of the EEG classifiers in MI-based BCIs.

Eye movements in response to different cognitive activities measured by eyetracking: a prospective study on some of the neurolinguistics programming theories.

The eyes are in constant movement to optimize the interpretation of the visual scene by the brain. Eye movements are controlled by complex neural networks that interact with the rest of the brain. The direction of our eye movements could thus be influenced by our cognitive activity (imagination, internal dialogue, memory, etc.). A given cognitive activity could then cause the gaze to move in a specific direction (a brief movement that would be instinctive and unconscious). Neuro Linguistic Programming (NLP), which was developed in the 1970s by Richard Bandler and John Grinder (psychologist and linguist respectively), issued a comprehensive theory associating gaze directions with specific mental tasks. According to this theory, depending on the visual path observed, one could go back to the participant's thoughts and cognitive processes. Although NLP is widely used in many disciplines (communication, psychology, psychotherapy, marketing, etc), to date, few scientific studies have examined the validity of this theory. Using eye tracking, this study explores one of the hypotheses of this theory, which is one of the pillars of NLP on visual language. We created a protocol based on a series of questions of different types (supposed to engage different brain areas) and we recorded by eye tracking the gaze movements at the end of each question while the participants were thinking and elaborating on the answer. Our results show that 1) complex questions elicit significantly more eye movements than control questions that necessitate little reflection, 2) the movements are not random but are oriented in selected directions, according to the different question types, 3) the orientations observed are not those predicted by the NLP theory. This pilot experiment paves the way for further investigations to decipher the close links between eye movements and neural network activities in the brain.

Feature optimization based on improved novel global harmony search algorithm for motor imagery electroencephalogram classification.

Effectively decoding electroencephalogram (EEG) pattern for specific mental tasks is a crucial topic in the development of brain-computer interface (BCI). Extracting common spatial pattern (CSP) features from motor imagery EEG signals is often highly dependent on the selection of frequency band and time interval. Therefore, optimizing frequency band and time interval would contribute to effective feature extraction and accurate EEG decoding. This study proposes an approach based on an improved novel global harmony search (INGHS) to optimize frequency-time parameters for effective CSP feature extraction. The INGHS algorithm is applied to find the optimal frequency band and temporal interval. The linear discriminant analysis and support vector machine are used for EEG pattern decoding. Extensive experimental studies are conducted on three EEG datasets to assess the effectiveness of our proposed method. The average test accuracy obtained by the time-frequency parameters selected by the proposed INGHS method is slightly better than artificial bee colony (ABC) and particle swarm optimization (PSO) algorithms. Furthermore, the INGHS algorithm is superior to PSO and ABC in running time. These superior experimental results demonstrate that the optimal frequency band and time interval selected by the INGHS algorithm could significantly improve the decoding accuracy compared with the traditional CSP method. This method has a potential to improve the performance of MI-based BCI systems.

Improving EEG Decoding via Clustering-Based Multitask Feature Learning.

Accurate electroencephalogram (EEG) pattern decoding for specific mental tasks is one of the key steps for the development of brain-computer interface (BCI), which is quite challenging due to the considerably low signal-to-noise ratio of EEG collected at the brain scalp. Machine learning provides a promising technique to optimize EEG patterns toward better decoding accuracy. However, existing algorithms do not effectively explore the underlying data structure capturing the true EEG sample distribution and, hence, can only yield a suboptimal decoding accuracy. To uncover the intrinsic distribution structure of EEG data, we propose a clustering-based multitask feature learning algorithm for improved EEG pattern decoding. Specifically, we perform affinity propagation-based clustering to explore the subclasses (i.e., clusters) in each of the original classes and then assign each subclass a unique label based on a one-versus-all encoding strategy. With the encoded label matrix, we devise a novel multitask learning algorithm by exploiting the subclass relationship to jointly optimize the EEG pattern features from the uncovered subclasses. We then train a linear support vector machine with the optimized features for EEG pattern decoding. Extensive experimental studies are conducted on three EEG data sets to validate the effectiveness of our algorithm in comparison with other state-of-the-art approaches. The improved experimental results demonstrate the outstanding superiority of our algorithm, suggesting its prominent performance for EEG pattern decoding in BCI applications.

A Brain-Computer Interface for Teleoperation of a Semiautonomous Mobile Robotic Assistive System Using SLAM

The proposed assistive hybrid brain-computer interface (BCI) semiautonomous mobile robotic arm demonstrates a design that is (1) adaptable by observing environmental changes with sensors and deploying alternate solutions and (2) versatile by receiving commands from the user’s brainwave signals through a noninvasive electroencephalogram cap. Composed of three integrated subsystems, a hybrid BCI controller, an omnidirectional mobile base, and a robotic arm, the proposed robot has commands mapped to the user’s brainwaves related to a set of specific physical or mental tasks. The implementation of sensors and the camera systems enable both the mobile base and the arm to be semiautonomous. The mobile base’s SLAM algorithm has obstacle avoidance capability and path planning to assist the robot maneuver safely. The robot arm calculates and deploys the necessary joint movement to pick up or drop off a desired object selected by the user via a brainwave controlled cursor on a camera feed. Validation, testing, and implementation of the subsystems were conducted using Gazebo. Communication between the BCI controller and the subsystems is tested independently. A loop of prerecorded brainwave data related to each specific task is used to ensure that the mobile base command is executed; the same prerecorded file is used to move the robot arm cursor and initiate a pick-up or drop-off action. A final system test is conducted where the BCI controller input moves the cursor and selects a goal point. Successful virtual demonstrations of the assistive robotic arm show the feasibility of restoring movement capability and autonomy for a disabled user.

Long Multi-Stage Training for a Motor-Impaired User in a BCI Competition

In a Mental Imagery Brain-Computer Interface the user has to perform a specific mental task that generates electroencephalography (EEG) components, which can be translated in commands to control a BCI system. The development of a high-performance MI-BCI requires a long training, lasting several weeks or months, in order to improve the ability of the user to manage his/her mental tasks. This works aims to present the design of a MI-BCI combining mental imaginary and cognitive tasks for a severely motor impaired user, involved in the BCI race of the Cybathlon event, a competition of people with severe motor disability. In the BCI-race, the user becomes a pilot in a virtual race game against up to three other pilots, in which each pilot has to control his/her virtual car by his/her mental tasks. We present all the procedures followed to realize an effective MI-BCI, from the user's first contact with a BCI technology to actually controlling a video-game through her EEG. We defined a multi-stage user-centered training protocol in order to successfully control a BCI, even in a stressful situation, such as that of a competition. We put a specific focus on the human aspects that influenced the long training phase of the system and the participation to the competition.

Learning deep features for task-independent EEG-based biometric verification

Abstract Considerable interest has been recently devoted to the exploitation of brain activity as biometric identifier in automatic recognition systems, with a major focus on data acquired through electroencephalography (EEG). Several researches have in fact confirmed the presence of discriminative characteristics within brain signals recorded while performing specific mental tasks. Yet, to make EEG-based recognition appealing for practical applications, it would be highly advisable to investigate the existence and permanence of such distinctive traits while performing different mental tasks. In this regard, the present study evaluates the feasibility of performing task-independent EEG-based biometric recognition. A deep learning approach using siamese convolutional neural networks is employed to extract, from the considered EEG recordings, subject-specific template representations. An extensive set of experimental tests, performed on a multi-session database comprising EEG data acquired from 45 subjects while performing six different tasks, is employed to evaluate whether it is actually possible to verify the identity of a subject using brain signals, regardless the performed mental task.

Success and failure of controlling the real-time functional magnetic resonance imaging neurofeedback signal are reflected in the striatum.

IntroductionOver the last decades, neurofeedback has been applied in variety of research contexts and therapeutic interventions. Despite this extensive use, its neural mechanisms are still under debate. Several scientific advances have suggested that different networks become jointly active during neurofeedback, including regions generally involved in self‐regulation, regions related to the specific mental task driving the neurofeedback and regions generally involved in feedback learning (Sitaram et al., 2017, Nature Reviews Neuroscience, 18, 86).MethodsTo investigate the neural mechanisms specific to neurofeedback but independent from general effects of self‐regulation, we compared brain activation as measured with functional magnetic resonance imaging (fMRI) across different mental tasks involving gradual self‐regulation with and without providing neurofeedback. Ten participants freely chose one self‐regulation task and underwent two training sessions during fMRI scanning, one with and one without receiving neurofeedback. During neurofeedback sessions, feedback signals were provided in real‐time based on activity in task‐related, individually defined target regions. In both sessions, participants aimed at reaching and holding low, medium, or high brain‐activation levels in the target region.ResultsDuring gradual self‐regulation with neurofeedback, a network of cortical control regions as well as regions implicated in reward and feedback processing were activated. Self‐regulation with feedback was accompanied by stronger activation within the striatum across different mental tasks. Additional time‐resolved single‐trial analysis revealed that neurofeedback performance was positively correlated with a delayed brain response in the striatum that reflected the accuracy of self‐regulation.ConclusionOverall, these findings support that neurofeedback contributes to self‐regulation through task‐general regions involved in feedback and reward processing.

Evaluating spiking neural models in the classification of motor imagery EEG signals using short calibration sessions

Since the emergence of brain computer interface (BCI), several methods have been applied to associate an electroencephalographic (EEG) recording with a specific mental task. Particularly, in the classification stage, several techniques such as linear Fisher discriminant (LD), feed-forward artificial neural networks (FNN) and radial basis function neural networks (RBF) have been applied successfully with BCI applications. However, in BCI applications, there is a challenge related to avoid long and tedious calibration session for users, this implies that the classification techniques used during the classification stage, have to be trained with a reduced number of EEG recordings. However, most of the classification techniques require several samples to learn accurately an association with a particular mental task. Since the spiking neural models (SNM) have shown their robustness in pattern recognition problems, this paper is focused on demonstrating that they are potential alternatives to classify EEG recordings when they are trained with a reduced number of data samples. To do that, we computed the coherence from a subset of three electrodes to obtain the feature vector of each EEG recording. Then, this information was classified using the SNM. In order to evaluate the robustness, the SNM was trained varying the number of samples. Furthermore, based on the performance and the confidence interval achieved in the classification, we developed two indexes to evaluate and compare the SNM against LD, FNN and RBF. The experimental results over the IIIa, IVa and V data sets from BCI International Competition III, suggest that the SNM are the best option to avoid long calibration sessions.

Predicting Mental Imagery-Based BCI Performance from Personality, Cognitive Profile and Neurophysiological Patterns.

Mental-Imagery based Brain-Computer Interfaces (MI-BCIs) allow their users to send commands to a computer using their brain-activity alone (typically measured by ElectroEncephaloGraphy—EEG), which is processed while they perform specific mental tasks. While very promising, MI-BCIs remain barely used outside laboratories because of the difficulty encountered by users to control them. Indeed, although some users obtain good control performances after training, a substantial proportion remains unable to reliably control an MI-BCI. This huge variability in user-performance led the community to look for predictors of MI-BCI control ability. However, these predictors were only explored for motor-imagery based BCIs, and mostly for a single training session per subject. In this study, 18 participants were instructed to learn to control an EEG-based MI-BCI by performing 3 MI-tasks, 2 of which were non-motor tasks, across 6 training sessions, on 6 different days. Relationships between the participants’ BCI control performances and their personality, cognitive profile and neurophysiological markers were explored. While no relevant relationships with neurophysiological markers were found, strong correlations between MI-BCI performances and mental-rotation scores (reflecting spatial abilities) were revealed. Also, a predictive model of MI-BCI performance based on psychometric questionnaire scores was proposed. A leave-one-subject-out cross validation process revealed the stability and reliability of this model: it enabled to predict participants’ performance with a mean error of less than 3 points. This study determined how users’ profiles impact their MI-BCI control ability and thus clears the way for designing novel MI-BCI training protocols, adapted to the profile of each user.

Classification of Mental States Based on Spatiospectral Patterns of Brain Electrical Activity

Abstract: Classification of human thought is an emerging research field that may allow us to understand human brainfunctions and further develop advanced brain-computer interface (BCI) systems. In the present study, we introducea new approach to classify various mental states from noninvasive electrophysiological recordings of human brainactivity. We utilized the full spatial and spectral information contained in the electroencephalography (EEG) signalsrecorded while a subject is performing a specific mental task. For this, the EEG data were converted into a 2D spa-tiospectral pattern map, of which each element was filled with 1, 0, and -1 reflecting the degrees of event-relatedsynchronization (ERS) and event-related desynchronization (ERD). We evaluated the similarity between a current(input) 2D pattern map and the template pattern maps (database), by taking the inner-product of pattern matrices.Then, the current 2D pattern map was assigned to a class that demonstrated the highest similarity value. For theverification of our approach, eight participants took part in the present study; their EEG data were recorded whilethey performed four different cognitive imagery tasks. Consistent ERS/ERD patterns were observed more frequentlybetween trials in the same class than th ose in different classes, indicating that th ese spatiospectral pattern maps couldbe used to classify different mental states. The classification accuracy was evaluated for each participant from boththe proposed approach and a conventional mental state classification method based on the inter-hemispheric spectralpower asymmetry, using the leave-one-out cross-validation (LOOCV). An average accuracy of 68.13% (± 9.64%) wasattained for the proposed method; whereas an average accuracy of 57% (± 5.68%) was attained for the conventionalmethod (significance was assessed by the one-tail paired t-test, p < 0.01), showing that the proposed simple clas-sification approach might be one of the promising methods in discriminating various mental states.Key words:brain-computer interface (BCI), spatiospectral pattern, mental state classification, mind reading, elec-troencephalography (EEG)

A motor imagery-based online interactive brain-controlled switch: Paradigm development and preliminary test

To develop a practical motor imagery-based brain-controlled switch as functional as a real-world switch that is reliable with a minimal false positive operation rate and convenient for users without the need of attention to the switch during a 'No Control' state (when not to activate the switch). Four healthy volunteers were instructed to perform an intended motor imagery task following an external sync signal in order to turn on a virtual switch provided on a computer screen. No specific mental task was required during the 'No Control' state. The beta band event-related frequency power (event-related desynchronization or ERD) from a single EEG Laplacian channel was monitored online in real-time. The computer continuously monitored the relative ERD power level until it exceeded a pre-set threshold and turned on the virtual switch. Subject 1 achieved lowest average false positive rate of 0.4+/-0.9% in a five-session online study during the entire 'No Control' state, whereby the subject required 6.8+/-0.6 s of active urging time or total response time of 20.5+/-1.9 s to perform repeated attempts in order to turn on the switch in the online interactive switch operation. The average false positive rate among four subjects was 0.8+/-0.4% with average active urging time of 12.3+/-4.4 s or average response time of 36.9+/-13.0 s. Offline analysis from subject 2 shows that the overall performance from 10-fold cross-validation was 96.2% with 3 consecutive epoch averaging, which was further improved to 99.0% by computationally intensive methods. The novel design of the brain-controlled switch using the ERD feature associated with motor imagery achieved minimal false positive rate with a reasonable active urging time or response time to activate the switch. The reliability and convenience of the developed brain-controlled switch may extend current brain-computer interface capacities in practical communication and control applications.

Plausibility assessment of a 2-state self-paced mental task-based BCI using the no-control performance analysis

The feasibility of having a self-paced brain–computer interface (BCI) based on mental tasks is investigated. The EEG signals of four subjects performing five mental tasks each are used in the design of a 2-state self-paced BCI. The output of the BCI should only be activated when the subject performs a specific mental task and should remain inactive otherwise. For each subject and each task, the feature coefficient and the classifier that yield the best performance are selected, using the autoregressive coefficients as the features. The classifier with a zero false positive rate and the highest true positive rate is selected as the best classifier. The classifiers tested include: linear discriminant analysis, quadratic discriminant analysis, Mahalanobis discriminant analysis, support vector machine, and radial basis function neural network. The results show that: (1) some classifiers obtained the desired zero false positive rate; (2) the linear discriminant analysis classifier does not yield acceptable performance; (3) the quadratic discriminant analysis classifier outperforms the Mahalanobis discriminant analysis classifier and performs almost as well as the radial basis function neural network; and (4) the support vector machine classifier has the highest true positive rates but unfortunately has nonzero false positive rates in most cases.

Evaluación de un protocolo de entrenamiento en una interfaz cerebro-computadora: propuesta de tareas mentales

A brain-computer interface (BCI) is based on the analysis of the electroencephalographic (EEG) signals recorded during certain mental activities, to control an external device. Main users are people with severe neuromuscular disorders, like amyotrophic lateral sclerosis. One of the most important problems to control a BCI is the need of providing suitable training, helping subjects to get some control of the EEG signals.To carry out a study of possible effects of the use of specific mental tasks during the first phase of the training period.Eighteen healthy untrained subjects took part in the experiment. A group of subjects were trained to discriminate between two motor imagery tasks (imagination of right and left hand movements). Another group were trained to discriminate between a motor imagery task (imagination of right hand movements) and mental relaxation. Objective and subjective measures based on questionnaires were taken.Some subjects do not achieved EEG control, but subjects at the second group showed a greater facility to control a BCI.Training protocols should not be randomly chosen; they must be adapted to the subject to be effective. Sometimes it is necessary to increase the number of sessions without feedback before submitting a subject to a session with feedback, and a correct choice of the mental tasks is very important. Mental tasks which are easy to discriminate improve classification result and produce better satisfaction to the subject.

EEG classification using generative independent component analysis

We present an application of independent component analysis (ICA) to the discrimination of mental tasks for EEG-based brain computer interface systems. ICA is most commonly used with EEG for artifact identification with little work on the use of ICA for direct discrimination of different types of EEG signals. By viewing ICA as a generative model, we can use Bayes’ rule to form a classifier. We fit spatial filters and source distribution parameters simultaneously and investigate whether these are sufficiently informative to produce good results when compared to more traditional methods based on using temporal features as inputs to off-the-shelf classifiers. Experiments suggest that state-of-the-art results may indeed be found without explicitly using temporal features. We extend the method to using a mixture of ICA models, consistent with the assumption that subjects may have more than one approach to thinking about a specific mental task.

Just one g: consistent results from three test batteries

The concept of a general intelligence factor or g is controversial in psychology. Although the controversy swirls at many levels, one of the most important involves g's identification and measurement in a group of individuals. If g is actually predictive of a range of intellectual performances, the factor identified in one battery of mental ability tests should be closely related to that identified in another dissimilar aggregation of abilities. We addressed the extent to which this prediction was true using three mental ability batteries administered to a heterogeneous sample of 436 adults. Though the particular tasks used in the batteries reflected varying conceptions of the range of human intellectual performance, the g factors identified by the batteries were completely correlated (correlations were .99, .99, and 1.00). This provides further evidence for the existence of a higher-level g factor and suggests that its measurement is not dependent on the use of specific mental ability tasks.

Shadows of Thoughts Revealed

W hat do thoughts look like in the human brain? Does every mental sentence, every intention, every little decision explode in a telltale pattern of neurochemical activity? If so, might a brain scan someday provide a printout of a person's most intimate thoughts? As of 1990, neither scientists nor philosophers have gained sufficient insight into the nature of cognition to peruse the private diaries nestled within the brain's convoluted folds. But researchers using new technology have now developed the first pictures of what they call shadows of Their approach can't detect the contents of thoughts. However, it clearly differentiates among many of the electrochemical fireworks that underlie specific acts of cognition. The technique's developers say the high-resolution images they obtain may soon provide a wealth of practical benefits ranging from improved assessment of brain injury to reduced incidence of fatigue-related industrial accidents. Moreover, the work offers intriguing new clues about how the brain perceives and constantly updates its sense of the world and its sense of self. Generations of neuroscientists have dreamed about having a movie camera for observing thinking in the says Alan Gevins, director of EEG Systems Laboratory, a private brain-research center in San Francisco. Revolutionary advances in computers are now making this possible. A space-age electroencephalography (EEG) device, worn like a soft helmet, is the heart of Gevins' mind-camera. Unlike old-fashioned EEG machines, which record electrical signals at 19 points on the scalp and enable clinicians to detect gross brain-wave abnormalities, Gevins' device records electrical activity at 124 points. Computers monitor the skullbound electrical squalls we call thought and map their shifting locations within a three-dimensional image of the monitored brain, created by a magnetic resonance imager (MRI). The EEG-MRI combo -called the mental activity network scanner, or MANSCAN snaps a fresh picture of brain activity every 4 milliseconds, providing far more images per minute than positron emission tomography (PET), another popular brain-imaging technique (SN: 2/25/89, p.116). Together, the string of snapshots provides a movie of neuronal bursts, tracing peaks and troughs of brain activity while the monitored person performs specific mental tasks. Speaking in St. Louis last week at the *~ :