Human Brain Activity Research Articles

Application of an effective neural network architecture based on deep learning algorithms for the development of a noninvasive neurocomputer interface

Abstract. The article highlights the relevance of the development of modern noninvasive neurocomputer interfaces and identifies a problem in their development, which is the low accuracy of decoding human brain activity using modern noninvasive bidirectional neurocomputer interfaces, which makes it difficult to develop fully functioning noninvasive neuroprostheses. This problem is associated with a small number of domestic research in this area, as well as with an insufficient number of necessary tools for the development of this kind of neuroprostheses. The paper presents the principle of operation of this kind of interfaces, as well as varieties of neural interfaces. The scope of application of neurointerfaces and possible prospects for the development of this field are considered. The need to develop an artificial neural network using fuzzy logic aimed at improving the efficiency of isolating and filtering subtle signal patterns and structures of the human brain from the general signal background is justified.

Mobile EEG for the study of cognitive-motor interference during swimming?

Research on brain function in natural environments has become a new interest in cognitive science. In this study, we aim to advance mobile electroencephalography (EEG) participant and device mobility. We investigated the feasibility of measuring human brain activity using mobile EEG during a full-body motion task as swimming, by the example of cognitive-motor interference (CMI). Eleven participants were given an auditory oddball task while sitting and swimming, with mobile EEG recording ongoing brain activity. Measures of interest were event-related potentials (ERPs) elicited by experimental stimuli. While the auditory N100 was measured to verify signal quality, the P300 to task-relevant stimuli served as a marker of CMI effects. Analyzes were first performed within subjects, while binomial tests assessed the proportion of significant effects. Event-related changes in the time-frequency domain around turns during swimming were analyzed in an exploratory fashion. The successful recording of the N100 in all conditions shows that the setup was functional throughout the experiment. Regarding CMI, we did not find reliable changes in P300 amplitude in different motor settings in all subjects. However, we found plausible modulations in the alpha/mu and beta bands before and after turns. This study shows that it is generally feasible to measure mobile EEG in the time and time-frequency domain in an aquatic environment while subjects are freely moving. We see promising potential in the use of mobile EEG in extreme settings, advancing toward the application of mobile EEG in more real-life situations.

A hybrid spatiotemporal deep belief network and sparse representation-based framework reveals multilevel core functional components in decoding multitask fMRI signals.

Decoding human brain activity on various task-based functional brain imaging data is of great significance for uncovering the functioning mechanism of the human mind. Currently, most feature extraction model-based methods for brain state decoding are shallow machine learning models, which may struggle to capture complex and precise spatiotemporal patterns of brain activity from the highly noisy fMRI raw data. Moreover, although decoding models based on deep learning methods benefit from their multilayer structure that could extract spatiotemporal features at multiscale, the relatively large populations of fMRI datasets are indispensable, and the explainability of their results is elusive. To address the above problems, we proposed a computational framework based on hybrid spatiotemporal deep belief network and sparse representations to differentiate multitask fMRI (tfMRI) signals. Using a relatively small cohort of tfMRI data as a test bed, our framework can achieve an average classification accuracy of 97.86% and define the multilevel temporal and spatial patterns of multiple cognitive tasks. Intriguingly, our model can characterize the key components for differentiating the multitask fMRI signals. Overall, the proposed framework can identify the interpretable and discriminative fMRI composition patterns at multiple scales, offering an effective methodology for basic neuroscience and clinical research with relatively small cohorts.

Category-Selective Representation of Relationships in the Visual Cortex.

Understanding social interaction requires processing social agents and their relationships. The latest results show that much of this process is visually solved: visual areas can represent multiple people encoding emergent information about their interaction that is not explained by the response to the individuals alone. A neural signature of this process is an increased response in visual areas, to face-to-face (seemingly interacting) people, relative to people presented as unrelated (back-to-back). This effect highlighted a network of visual areas for representing relational information. How is this network organized? Using functional MRI, we measured the brain activity of healthy female and male humans (N = 42), in response to images of two faces or two (head-blurred) bodies, facing toward or away from each other. Taking the facing > non-facing effect as a signature of relation perception, we found that relations between faces and between bodies were coded in distinct areas, mirroring the categorical representation of faces and bodies in the visual cortex. Additional analyses suggest the existence of a third network encoding relations between (nonsocial) objects. Finally, a separate occipitotemporal network showed the generalization of relational information across body, face, and nonsocial object dyads (multivariate pattern classification analysis), revealing shared properties of relations across categories. In sum, beyond single entities, the visual cortex encodes the relations that bind multiple entities into relationships; it does so in a category-selective fashion, thus respecting a general organizing principle of representation in high-level vision. Visual areas encoding visual relational information can reveal the processing of emergent properties of social (and nonsocial) interaction, which trigger inferential processes.

Enhancing EEG-based cross-day mental workload classification using periodic component of power spectrum

Objective. The day-to-day variability of electroencephalogram (EEG) poses a significant challenge to decode human brain activity in EEG-based passive brain-computer interfaces (pBCIs). Conventionally, a time-consuming calibration process is required to collect data from users on a new day to ensure the performance of the machine learning-based decoding model, which hinders the application of pBCIs to monitor mental workload (MWL) states in real-world settings. Approach. This study investigated the day-to-day stability of the raw power spectral density (PSD) and their periodic and aperiodic components decomposed by the Fitting Oscillations and One-Over-F algorithm. In addition, we validated the feasibility of using periodic components to improve cross-day MWL classification performance. Main results. Compared to the raw PSD (69.9% ± 18.5%) and the aperiodic component (69.4% ± 19.2%), the periodic component had better day-to-day stability and significantly higher cross-day classification accuracy (84.2% ± 11.0%). Significance. These findings indicate that periodic components of EEG have the potential to be applied in decoding brain states for more robust pBCIs.

Reconstructing visual illusory experiences from human brain activity.

Visual illusions provide valuable insights into the brain's interpretation of the world given sensory inputs. However, the precise manner in which brain activity translates into illusory experiences remains largely unknown. Here, we leverage a brain decoding technique combined with deep neural network (DNN) representations to reconstruct illusory percepts as images from brain activity. The reconstruction model was trained on natural images to establish a link between brain activity and perceptual features and then tested on two types of illusions: illusory lines and neon color spreading. Reconstructions revealed lines and colors consistent with illusory experiences, which varied across the source visual cortical areas. This framework offers a way to materialize subjective experiences, shedding light on the brain's internal representations of the world.

Phase Synchronization and Dynamic Behavior of a Novel Small Heterogeneous Coupled Network

Studying the firing dynamics and phase synchronization behavior of heterogeneous coupled networks helps us understand the mechanism of human brain activity. In this study, we propose a novel small heterogeneous coupled network in which the 2D Hopfield neural network (HNN) and the 2D Hindmarsh–Rose (HR) neuron are coupled through a locally active memristor. The simulation results show that the network exhibits complex dynamic behavior and is different from the usual phase synchronization. More specifically, the membrane potential of the 2D HR neuron exhibits five stable firing modes as the coupling parameter k1 changes. In addition, it is found that in the local region of k1, the number of spikes in bursting firing increases with the increase in k1. More interestingly, the network gradually changes from synchronous to asynchronous during the increase in the coupling parameter k1 but suddenly becomes synchronous around the coupling parameter k1 = 1.96. As far as we know, this abnormal synchronization behavior is different from the existing findings. This research is inspired by the fact that the episodic synchronous abnormal firing of excitatory neurons in the hippocampus of the brain can lead to diseases such as epilepsy. This helps us further understand the mechanism of brain activity and build bionic systems. Finally, we design the simulation circuit of the network and implement it on an STM32 microcontroller.

Mental image reconstruction from human brain activity: Neural decoding of mental imagery via deep neural network-based Bayesian estimation

Visual images observed by humans can be reconstructed from their brain activity. However, the visualization (externalization) of mental imagery is challenging. Only a few studies have reported successful visualization of mental imagery, and their visualizable images have been limited to specific domains such as human faces or alphabetical letters. Therefore, visualizing mental imagery for arbitrary natural images stands as a significant milestone. In this study, we achieved this by enhancing a previous method. Specifically, we demonstrated that the visual image reconstruction method proposed in the seminal study by Shen et al. (2019) heavily relied on low-level visual information decoded from the brain and could not efficiently utilize the semantic information that would be recruited during mental imagery. To address this limitation, we extended the previous method to a Bayesian estimation framework and introduced the assistance of semantic information into it. Our proposed framework successfully reconstructed both seen images (i.e., those observed by the human eye) and imagined images from brain activity. Quantitative evaluation showed that our framework could identify seen and imagined images highly accurately compared to the chance accuracy (seen: 90.7%, imagery: 75.6%, chance accuracy: 50.0%). In contrast, the previous method could only identify seen images (seen: 64.3%, imagery: 50.4%). These results suggest that our framework would provide a unique tool for directly investigating the subjective contents of the brain such as illusions, hallucinations, and dreams.

Prestimulus alpha power signals attention to retrieval.

The human brain is in distinct processing modes at different times. Specifically, a distinction can be made between encoding and retrieval modes, which refer to the brain's state when it is storing new information or searching for old information, respectively. Recent research proposed the idea of a "ready-to-encode" mode, which describes a prestimulus effect in brain activity that signals (external) attention to encoding and predicts subsequent memory performance. Whether there is also a corresponding "ready-to-retrieve" mode in human brain activity is currently unclear. In this study, we examined whether prestimulus oscillations can be linked to (internal) attention to retrieval. We show that task cues to prepare for retrieval (or testing) in comparison with restudy of previously studied vocabulary word pairs led to a significant decrease of prestimulus alpha power just before the onset of word stimuli. Beamformer analysis localized this effect in the right secondary visual cortex (Brodmann area 18). Correlation analysis showed that the task cue-induced, prestimulus alpha power effect is positively related to stimulus-induced alpha/beta power, which in turn predicted participants' memory performance. The results are consistent with the idea that prestimulus alpha power signals internal attention to retrieval, which promotes the elaborative processing of episodic memories. Future research on brain-computer interfaces may find the findings interesting regarding the potential of using online measures of fluctuating alpha oscillations to trigger the presentation and sequencing of restudy and testing trials, ultimately enhancing instructional learning strategies.

Functional diversity of visual cortex improves constraint-free natural image reconstruction from human brain activity

Previous brain decoding studies using functional magnetic resonance imaging (fMRI) have greatly advanced our understanding of human visual coding and non-invasive brain-machine interfaces. However, most of these studies focus on classifying a limited number of image categories or reconstructing visual images with additional information, e.g., semantic categories and textual cues. Constraint-free visual reconstruction remains scarce. Here, we propose a generative network based on the functional diversity of the human visual cortex (FDGen) that takes multivariate brain activity as input and directly reconstructs natural images perceived by observers without any additional cues (semantic categories or textual description). Our FDGen is augmented by two bio-inspired computational modules. Based on the functional specializations of the human visual cortex, we propose a new function-based input module (FIM) that projects responses from different brain regions into separate feature spaces. Second, inspired by human attention, we construct a computational module to derive attentive feature weights at the function level to refine the feature map. These function-selection modules (FSMs) allow the network to dynamically select multiscale visual information during the generation process. We test FDGen on the popular fMRI datasets of natural images and achieve highly robust performance. Our work represents an important step forward in the development of fMRI-based brain decoding algorithms and highlights the utility of neuroscience theories in the design of deep learning models.

Modeling naturalistic face processing in humans with deep convolutional neural networks.

Deep convolutional neural networks (DCNNs) trained for face identification can rival and even exceed human-level performance. The ways in which the internal face representations in DCNNs relate to human cognitive representations and brain activity are not well understood. Nearly all previous studies focused on static face image processing with rapid display times and ignored the processing of naturalistic, dynamic information. To address this gap, we developed the largest naturalistic dynamic face stimulus set in human neuroimaging research (700+ naturalistic video clips of unfamiliar faces). We used this naturalistic dataset to compare representational geometries estimated from DCNNs, behavioral responses, and brain responses. We found that DCNN representational geometries were consistent across architectures, cognitive representational geometries were consistent across raters in a behavioral arrangement task, and neural representational geometries in face areas were consistent across brains. Representational geometries in late, fully connected DCNN layers, which are optimized for individuation, were much more weakly correlated with cognitive and neural geometries than were geometries in late-intermediate layers. The late-intermediate face-DCNN layers successfully matched cognitive representational geometries, as measured with a behavioral arrangement task that primarily reflected categorical attributes, and correlated with neural representational geometries in known face-selective topographies. Our study suggests that current DCNNs successfully capture neural cognitive processes for categorical attributes of faces but less accurately capture individuation and dynamic features.

Seasonal variations of functional connectivity of human brains

Seasonal variations have long been observed in various aspects of human life. While there is an abundance of research that has characterized seasonality effects in, for example, cognition, mood, and behavior, including studies of underlying biophysical mechanisms, direct measurements of seasonal variations of brain functional activities have not gained wide attention. We have quantified seasonal effects on functional connectivity as derived from MRI scans. A cohort of healthy human subjects was divided into four groups based on the seasons of their scanning dates as documented in the image database of the Human Connectome Project. Sinusoidal functions were used as regressors to determine whether there were significant seasonal variations in measures of brain activities. We began with the analysis of seasonal variations of the fractional amplitudes of low frequency fluctuations of regional functional signals, followed by the seasonal variations of functional connectivity in both global- and network-level. Furthermore, relevant environmental factors, including average temperature and daylength, were found to be significantly associated with brain functional activities, which may explain how the observed seasonal fluctuations arise. Finally, topological properties of the brain functional network also showed significant variations across seasons. All the observations accumulated revealed seasonality effects of human brain activities in a resting-state, which may have important practical implications for neuroimaging research.

Dissociated neural tracking as an index of speech understanding in background noise

Human brain activity has been shown to track hierarchical linguistic units embedded in connected speech and these responses can be directly modulated by changes in speech intelligibility caused by spectral degradation or prior knowledge. In this study, we introduce a background noise and manipulate its level relative to the target speech to test the hypothesis that the tracking responses are modulated differently by the variations in Signal-to-Noise Ratio (SNR) and changes in speech intelligibility. Electroencephalography (EEG) responses to speech in quiet and in noise (multi-talker babble) at three different levels were measured from 19 normal hearing participants. Driven by the reduction in intelligibility, cortical coherence to “abstract” linguistic units with no accompanying acoustic cues was reduced relative to the “speech only” condition, and lateralized to single cerebral hemispheres. In contrast, brain responses coherent to words, aligned with acoustic onsets, were bilateral and reduced systematically as noise level increased. Strength of the tracking response correlated with subjective ratings from each participant on how much they can understand the speech sentences at all different linguistic levels. These results provide an objective and sensitive neural marker of speech intelligibility which can be further developed into clinical applications for objective assessment of speech-in-noise understanding.

Optimizing network neuroscience computation of individual differences in human spontaneous brain activity for test-retest reliability.

A rapidly emerging application of network neuroscience in neuroimaging studies has provided useful tools to understand individual differences in intrinsic brain function by mapping spontaneous brain activity, namely intrinsic functional network neuroscience (ifNN). However, the variability of methodologies applied across the ifNN studies-with respect to node definition, edge construction, and graph measurements-makes it difficult to directly compare findings and also challenging for end users to select the optimal strategies for mapping individual differences in brain networks. Here, we aim to provide a benchmark for best ifNN practices by systematically comparing the measurement reliability of individual differences under different ifNN analytical strategies using the test-retest design of the Human Connectome Project. The results uncovered four essential principles to guide ifNN studies: (1) use a whole brain parcellation to define network nodes, including subcortical and cerebellar regions; (2) construct functional networks using spontaneous brain activity in multiple slow bands; and (3) optimize topological economy of networks at individual level; and (4) characterize information flow with specific metrics of integration and segregation. We built an interactive online resource of reliability assessments for future ifNN (https://ibraindata.com/research/ifNN).

Convolutional gated recurrent unit-driven multidimensional dynamic graph neural network for subject-independent emotion recognition

Electroencephalogram (EEG) could directly reflect human brain activities. Recently, EEG-based emotion recognition technology has attracted widespread attention. However, although most of the existing methods can extract temporal and spatial features of EEG signals, the extraction process of the time-series information and spatial features is separate from each other. Some spatial features may be lost when extracting time-series information, and vice versa. Additionally, the non-Euclidean representations between the EEG electrodes cannot be ignored. In response to these issues, we construct a convolutional gated recurrent unit-driven multidimensional dynamic graph neural network (CGRU-MDGN), which mines its local spatial characteristics of EEG signals while learning the temporal information and captures non-Euclidean spatial features between EEG channels to classify emotions more accurately. Specifically, we substitute the fully connected layers of the GRU with convolution layers, which can take into account both the local spatial features and time-series information of EEG signals. Next, multiple non-Euclidean spatial representations are generated using multidimensional dynamic graph convolution. Furthermore, the characteristics of different dimensions or views are merged through the adaptive-weighted summation technique to realize the maximized feature extraction as much as possible. Comprehensive experiments reveal that the constructed CGRU-MDGN could achieve competitive recognition results on subject-independent emotion recognition tasks.

Exploring EEG characteristics of multi-level mental stress based on human–machine system

Objective. The understanding of cognitive states is important for the development of human–machine systems (HMSs), and one of the fundamental but challenging issues is the understanding and assessment of the operator’s mental stress state in real task scenarios. Approach. In this paper, a virtual unmanned vehicle (UAV) driving task with multi-challenge-level was created to explore the operator’s mental stress, and the human brain activity during the task was tracked in real time via electroencephalography (EEG). A mental stress analysis dataset for the virtual UAV task was then developed and used to explore the neural activation patterns associated with mental stress activity. Finally, a multiple attention-based convolutional neural network (MACN) was constructed for automatic stress assessment using the extracted stress-sensitive neural activation features. Main Results. The statistical results of EEG power spectral density (PSD) showed that frontal theta-PSD decreased with increasing task difficulty, and central beta-PSD increased with increasing task difficulty, indicating that neural patterns showed different trends under different levels of mental stress. The performance of the proposed MACN was evaluated based on the dimensional model, and results showed that average three-class classification accuracies of 89.49%/89.88% were respectively achieved for arousal/valence. Significance. The results of this paper suggest that objective assessment of mental stress in a HMS based on a virtual UAV scenario is feasible, and the proposed method provides a promising solution for cognitive computing and applications in human–machine tasks.

Successful alpha neurofeedback training enhances working memory updating and event-related potential activity

Neurofeedback (NF) is a promising method to self-regulate human brain activity for cognition enhancement. Due to the unclear results of alpha NF training on working memory updating as well as the impact of feedback modality on NF learning, this study aimed to understand further the underlying neural mechanism of alpha NF training effects on working memory updating, where the NF learning was also compared between visual and auditory feedback modalities. A total of 30 participants were assigned to Visual NF, Auditory NF, and Control groups. Working memory updating was evaluated by n-back (n =2,3) tasks before and after five alpha upregulation NF sessions. The result showed no significant difference in NF learning performance between the Visual and Auditory groups, indicating that the difference in feedback modality did not affect NF learning. In addition, compared to the control group, the participants who achieved successful NF learning showed a significant increase in n-back behavioral performance and P3a amplitude in 2-back and a significant decrease in P3a latency in 3-back. Our results in n-back further suggested that successful alpha NF training might improve updating performance in terms of the behavioral and related event-related potential (ERP) measures. These findings contribute to the understanding of the effect of alpha training on memory updating and the design of NF experimental protocol in terms of feedback modality selection.

DMCA-GAN: Dual Multilevel Constrained Attention GAN for MRI-Based Hippocampus Segmentation.

Precise segmentation of the hippocampus is essential for various human brain activity and neurological disorder studies. To overcome the small size of the hippocampus and the low contrast of MR images, a dual multilevel constrained attention GAN for MRI-based hippocampus segmentation is proposed in this paper, which is used to provide a relatively effective balance between suppressing noise interference and enhancing feature learning. First, we design the dual-GAN backbone to effectively compensate for the spatial information damage caused by multiple pooling operations in the feature generation stage. Specifically, dual-GAN performs joint adversarial learning on the multiscale feature maps at the end of the generator, which yields an average Dice coefficient (DSC) gain of 5.95% over the baseline. Next, to suppress MRI high-frequency noise interference, a multilayer information constraint unit is introduced before feature decoding, which improves the sensitivity of the decoder to forecast features by 5.39% and effectively alleviates the network overfitting problem. Then, to refine the boundary segmentation effects, we construct a multiscale feature attention restraint mechanism, which forces the network to concentrate more on effective multiscale details, thus improving the robustness. Furthermore, the dual discriminators D1 and D2 also effectively prevent the negative migration phenomenon. The proposed DMCA-GAN obtained a DSC of 90.53% on the Medical Segmentation Decathlon (MSD) dataset with tenfold cross-validation, which is superior to the backbone by 3.78%.

Electroencephalogram-Based Subject Matching Learning (ESML): A Deep Learning Framework on Electroencephalogram-Based Biometrics and Task Identification.

An EEG signal (Electroencephalogram) is a bioelectric phenomenon reflecting human brain activities. In this paper, we propose a novel deep learning framework ESML (EEG-based Subject Matching Learning) using raw EEG signals to learn latent representations for EEG-based user identification and tack classification. ESML consists of two parts: one is the ESML1 model via an LSTM-based method for EEG-user linking, and one is the ESML2 model via a CNN-based method for EEG-task linking. The new model ESML is simple, but effective and efficient. It does not require any restrictions for EEG data collection on motions and thinking for users, and it does not need any EEG preprocessing operations, such as EEG denoising and feature extraction. The experiments were conducted on three public datasets and the results show that ESML performs the best and achieves significant performance improvement when compared to baseline methods (i.e., SVM, LDA, NN, DTS, Bayesian, AdaBoost and MLP). The ESML1 model provided the best precision at 96% with 109 users and the ESML2 model achieved 99% precision at 3-Class task classification. These experimental results provide direct evidence that EEG signals can be used for user identification and task classification.

Build a bridge between ECG and EEG signals for atrial fibrillation diagnosis using AI methods

Atrial fibrillation (AF) is a very common type of cardiac arrhythmia. The main characteristic of AF is an abnormally rapid and disordered atrial rhythm causing an atrial dysfunction, which can be visualized on an electrocardiograph (ECG) and distinguished by irregular fluctuations. Despite continuous and considerable efforts to analyze the pathophysiology of AF, it is challenging to determine the underlying pathogenesis of the disease in individual patients. This study aims to build a bridge between ECG and electroencephalogram (EEG) signals to probe the strong influence between human brain activity and AF by AI methods. We first found that the one-second data fragment shows the most excellent performance in our time window configuration. Secondly, in our proposed measurement, most cortical potentials were partly associated with AF. Thirdly, we found that only a few channels of data were sufficient for analysis. Finally, our experiment shows δ wave has the best performance compared to other wave bands. By AI methods, the paper contributes to concluding that δ wave band of EEG is the most associated brain wave type with AF. By EEG signals from typical regions, the central region, parietal and Occipital might be the most associated encephalic regions with AF. The clinical trial registration number for our study is ChiCTR2300068625.