Neural Representation Research Articles

A 7T fMRI investigation of hand and tool areas in the lateral and ventral occipitotemporal cortex.

Previous studies demonstrated the existence of hand and tool areas in lateral and ventral occipitotemporal cortex (OTC), as well as an overlap between them. We reinvestigated this organization using 7T fMRI, benefiting from a higher signal-to-noise ratio than 3T. This enabled us to include a wider array of categories to achieve a more holistic perspective, and to omit certain spatial preprocessing steps. Despite these improvements, univariate analysis confirmed the existence of hand-tool overlap across OTC, which is striking given the omission of the spatial preprocessing steps that can influence overlap. There was significantly more overlap between hands and tools, compared to other overlap types in the left hemisphere of OTC. The overlap was also larger in the left lateral OTC as compared to the right lateral OTC. We found in all hand areas a differentiation between tools and other types of objects, although they still responded more to bodies than to tools. Regarding the tool areas, we observed a differentiation between hands and other categories such as faces and non-tool objects. Left hemisphere tool areas also differentiated between hands and bodies. When excluding the overlapping voxels from the hand and tool areas, they still showed a significant response to tools or hands (compared to objects or faces) respectively. Multi-voxel pattern analysis indicated that neural representations in the hand areas showed greater similarity between hands and tools than between hands and other objects. In the tool areas, the neural representations between tools and hands and between tools and other type of objects were all equally similar. To summarize, capitalizing on the benefits of 7T fMRI, we further substantiate the evidence in favor of hand-tool overlap in several regions of occipitotemporal cortex.

Lower Limb Motion Recognition Based on sEMG and CNN-TL Fusion Model

To enhance the classification accuracy of lower limb movements, a fusion recognition model integrating a surface electromyography (sEMG)-based convolutional neural network, transformer encoder, and long short-term memory network (CNN-Transformer-LSTM, CNN-TL) was proposed in this study. By combining these advanced techniques, significant improvements in movement classification were achieved. Firstly, sEMG data were collected from 20 subjects as they performed four distinct gait movements: walking upstairs, walking downstairs, walking on a level surface, and squatting. Subsequently, the gathered sEMG data underwent preprocessing, with features extracted from both the time domain and frequency domain. These features were then used as inputs for the machine learning recognition model. Finally, based on the preprocessed sEMG data, the CNN-TL lower limb action recognition model was constructed. The performance of CNN-TL was then compared with that of the CNN, LSTM, and SVM models. The results demonstrated that the accuracy of the CNN-TL model in lower limb action recognition was 3.76%, 5.92%, and 14.92% higher than that of the CNN-LSTM, CNN, and SVM models, respectively, thereby proving its superior classification performance. An effective scheme for improving lower limb motor function in rehabilitation and assistance devices was thus provided.

Registered Report Stage II: Decoding the category information from evoked potentials to visible and invisible visual objects

Previous studies that use decoding methods and EEG to investigate the neural representation of the category information of visual objects focused mainly on consciously processed visual objects. It remains unclear whether the category information of unconsciously processed visual objects can be decoded and whether the decoding performance is different for consciously and unconsciously processed visual objects. The present study compared the neural decoding of the animacy category of visible and invisible visual objects via EEG and decoding methods. The results revealed that the animacy of visible visual objects could be decoded above the chance level by the P200, N300, and N400, but not by the early N/P100. However, the animacy of invisible visual objects could not be decoded above the chance level by neither early nor late ERP components. The decoding accuracy was greater for visible visual objects than that for invisible visual objects for the P200, N300 and N400. These results suggested that access to animacy category information for visual objects requires conscious processing.

Cracking the Code of the Jugular Vagal Sensory Neurons in Allergic Airway Responsiveness

Cracking the Code of the Jugular Vagal Sensory Neurons in Allergic Airway Responsiveness

Functional Connectivity Encodes Sound Locations by Lateralization Angles.

The ability to localize sound sources rapidly allows human beings to efficiently understand the surrounding environment. Previous studies have suggested that there is an auditory "where" pathway in the cortex for processing sound locations. The neural activation in regions along this pathway encodes sound locations by opponent hemifield coding, in which each unilateral region is activated by sounds coming from the contralateral hemifield. However, it is still unclear how these regions interact with each other to form a unified representation of the auditory space. In the present study, we investigated whether functional connectivity in the auditory "where" pathway encoded sound locations during passive listening. Participants underwent functional magnetic resonance imaging while passively listening to sounds from five distinct horizontal locations (-90°, -45°, 0°, 45°, 90°). We were able to decode sound locations from the functional connectivity patterns of the "where" pathway. Furthermore, we found that such neural representation of sound locations was primarily based on the coding of sound lateralization angles to the frontal midline. In addition, whole-brain analysis indicated that functional connectivity between occipital regions and the primary auditory cortex also encoded sound locations by lateralization angles. Overall, our results reveal a lateralization-angle-based representation of sound locations encoded by functional connectivity patterns, which could add on the activation-based opponent hemifield coding to provide a more precise representation of the auditory space.

Neural encoding of melodic expectations in music across EEG frequency bands.

The human brain tracks regularities in the environment and extrapolates these to predict future events. Prior work on music cognition suggests that low-frequency (1-8Hz) brain activity encodes melodic predictions beyond the stimulus acoustics. Building on this work, we aimed to disentangle the frequency-specific neural dynamics linked to melodic prediction uncertainty (modelled as entropy) and prediction error (modelled as surprisal) for temporal (note onset) and content (note pitch) information. By using multivariate temporal response function (TRF) models, we re-analysed the electroencephalogram (EEG) from 20 subjects (10 musicians) who listened to Western tonal music. Our results show that melodic expectation metrics improve the EEG reconstruction accuracy in all frequency bands below the gamma range (< 30 Hz). Crucially, we found that entropy contributed more strongly to the reconstruction accuracy enhancement compared to surprisal in all frequency bands. Additionally, we found that the encoding of temporal, but not content, information metrics was not limited to low frequencies, rather it extended to higher frequencies (> 8Hz). An analysis of the TRF weights revealed that the temporal predictability of a note (entropy of note onset) may be encoded in the delta- (1-4Hz) and beta-band (12-30 Hz) brain activity prior to the stimulus, suggesting that these frequency bands associate with temporal predictions. Strikingly, we also revealed that melodic expectations selectively enhanced EEG reconstruction accuracy in the beta band for musicians, and in the alpha band (8-12 Hz) for non-musicians, suggesting that musical expertise influences the neural dynamics underlying predictive processing in music cognition.

Towards Human Sensory Augmentation: A Cognitive Neuroscience Framework for Evaluating Integration of New Signals within Perception, Brain Representations, and Subjective Experience

New wearable devices and technologies provide unprecedented scope to augment or substitute human perceptual abilities. However, the flexibility to reorganize brain processing to use novel sensory signals during early sensitive periods in infancy is much less evident at later ages, making integration of new signals into adults’ perception a significant challenge. We believe that an approach informed by cognitive neuroscience is crucial for maximizing the true potential of new sensory technologies. Here, we present a framework for measuring and evaluating the extent to which new signals are integrated within existing structures of perception and experience. As our testbed, we use laboratory tasks in which healthy volunteers learn new, augmented perceptual-motor skills. We describe a suite of measures of (i) perceptual function (psychophysics), (ii) neural representations (fMRI/decoding), and (iii) subjective experience (qualitative interview/micro-phenomenology) targeted at testing hypotheses about how newly learned signals become integrated within perception and experience. As proof of concept, we provide example data showing how this approach allows us to measure changes in perception, neural processing, and subjective experience. We argue that this framework, in concert with targeted approaches to optimizing training and learning, provides the tools needed to develop and optimize new approaches to human sensory augmentation and substitution.

Neural Surfel Reconstruction: Addressing Loop Closure Challenges in Large-Scale 3D Neural Scene Mapping

Efficiently reconstructing complex and intricate surfaces at scale remains a significant challenge in 3D surface reconstruction. Recently, implicit neural representations have become a popular topic in 3D surface reconstruction. However, how to handle loop closure and bundle adjustment is a tricky problem for neural methods, because they learn the neural parameters globally. We present an algorithm that leverages the concept of surfels and expands relevant definitions to address such challenges. By integrating neural descriptors with surfels and framing surfel association as a deformation graph optimization problem, our method is able to effectively perform loop closure detection and loop correction in challenging scenarios. Furthermore, the surfel-level representation simplifies the complexity of 3D neural reconstruction. Meanwhile, the binding of neural descriptors to corresponding surfels produces a dense volumetric signed distance function (SDF), enabling the mesh reconstruction. Our approach demonstrates a significant improvement in reconstruction accuracy, reducing the average error by 16.9% compared to previous methods, while also generating modeling files that are up to 90% smaller than those produced by traditional methods.

Neural fields for rapid aircraft aerodynamics simulations

This paper presents a methodology to learn surrogate models of steady state fluid dynamics simulations on meshed domains, based on Implicit Neural Representations (INRs). The proposed models can be applied directly to unstructured domains for different flow conditions, handle non-parametric 3D geometric variations, and generalize to unseen shapes at test time. The coordinate-based formulation naturally leads to robustness with respect to discretization, allowing an excellent trade-off between computational cost (memory footprint and training time) and accuracy. The method is demonstrated on two industrially relevant applications: a RANS dataset of the two-dimensional compressible flow over a transonic airfoil and a dataset of the surface pressure distribution over 3D wings, including shape, inflow condition, and control surface deflection variations. On the considered test cases, our approach achieves a more than three times lower test error and significantly improves generalization error on unseen geometries compared to state-of-the-art Graph Neural Network architectures. Remarkably, the method can drastically accelerate high fidelity numerical solver, achieving a five order of magnitude speedup on the RANS transonic airfoil dataset.

Emerging many-to-one weighted mapping in hippocampus-amygdala network underlies memory formation

Memories are crucial for daily life, yet the network-level organizing principles governing neural representations of experiences remain unknown. Employing dual-site in vivo recording in freely behaving male mice, here we show that hippocampal dorsal CA1 (dCA1) and basolateral amygdala (BLA) utilize distinct coding strategies for novel experiences. A small assembly of BLA neurons emerged active during memory acquisition and persisted through consolidation, whereas most dCA1 neurons were engaged in both processes. Machine learning decoding revealed that dCA1 population spikes predicted BLA assembly firing rate, suggesting that most dCA1 neurons concurrently index an episodic event by rapidly establishing weighted communication with a specific BLA assembly – a process we term “many-to-one weighted mapping.” We also found that dCA1 reactivations preceded BLA assembly activity preferably during elongated and enlarged dCA1 ripples. Using a closed-loop strategy, we demonstrated that suppressing BLA activity after large dCA1 ripples impaired memory. These findings highlight a many-to-one weighted mapping mechanism underlying both the acquisition and consolidation of new memories.

Spatiotemporal implicit neural representation as a generalized traffic data learner

Spatiotemporal Traffic Data (STTD) measures the complex dynamical behaviors of the multiscale transportation system. Existing methods aim to reconstruct STTD using low-dimensional models. However, they are limited to data-specific dimensions or source-dependent patterns, restricting them from unifying representations. Here, we present a novel paradigm to address the STTD learning problem by parameterizing STTD as an implicit neural representation. To discern the underlying dynamics in low-dimensional regimes, coordinate-based neural networks that can encode high-frequency structures are employed to directly map coordinates to traffic variables. To unravel the entangled spatial–temporal interactions, the variability is decomposed into separate processes. We further enable modeling in irregular spaces such as sensor graphs using spectral embedding. Through continuous representations, our approach enables the modeling of a variety of STTD with a unified input, thereby serving as a generalized learner of the underlying traffic dynamics. It is also shown that it can learn implicit low-rank priors and smoothness regularization from the data, making it versatile for learning different dominating data patterns. We validate its effectiveness through extensive experiments in real-world scenarios, showcasing applications from corridor to network scales. Empirical results not only indicate that our model has significant superiority over conventional low-rank models, but also highlight that the versatility of the approach extends to different data domains, output resolutions, and network topologies. Comprehensive model analyses provide further insight into the inductive bias of STTD. We anticipate that this pioneering modeling perspective could lay the foundation for universal representation of STTD in various real-world tasks. PyTorch implementations of this project is publicly available at:https://github.com/tongnie/traffic_dynamics.

A universal hippocampal memory code across animals and environments.

How learning is affected by context is a fundamental question of neuroscience, as the ability to generalize learning to different contexts is necessary for navigating the world. An example of swift contextual generalization is observed in conditioning tasks, where performance is quickly generalized from one context to another. A key question in identifying the neural substrate underlying this ability is how the hippocampus (HPC) represents task-related stimuli across different environments, given that HPC cells exhibit place-specific activity that changes across contexts (remapping). In this study, we used calcium imaging to monitor hippocampal neuron activity as animals performed a conditioning task across multiple spatial contexts. We investigated whether hippocampal cells, which encode both spatial locations (place cells) and task-related information, could maintain their task representation even when their spatial encoding remapped in a new spatial context. To assess the consistency of task representations, we used advanced dimensionality reduction techniques combined with machine learning to develop manifold representations of population level HPC activity. The results showed that task-related neural representations remained stable even as place cell representations of spatial context changed, thus demonstrating similar embedding geometries of neural representations of the task across different spatial contexts. Notably, these patterns were not only consistent within the same animal across different contexts but also significantly similar across different animals, suggesting a standardized neural encoding or 'neural syntax' in the hippocampus. These findings bridge a critical gap between memory and navigation research, revealing how the hippocampus maintains cognitive consistency across different spatial environments. These findings also suggest that hippocampal function is governed by a neural framework shared between animals, an observation that may have broad implications for understanding memory, learning, and related cognitive processes. Looking ahead, this work opens new avenues for exploring the fundamental principles underlying hippocampal encoding strategies.

Childhood neglect is associated with alterations in neural prediction error signaling and the response to novelty.

One in eight children experience early life stress (ELS), which increases risk for psychopathology. ELS, particularly neglect, has been associated with reduced responsivity to reward. However, little work has investigated the computational specifics of this disrupted reward response - particularly with respect to the neural response to Reward Prediction Errors (RPE) - a critical signal for successful instrumental learning - and the extent to which they are augmented to novel stimuli. The goal of the current study was to investigate the associations of abuse and neglect, and neural representation of RPE to novel and non-novel stimuli. One hundred and seventy-eight participants (aged 10-18, M = 14.9, s.d. = 2.38) engaged in the Novelty task while undergoing functional magnetic resonance imaging. In this task, participants learn to choose novel or non-novel stimuli to win monetary rewards varying from $0 to $0.30 per trial. Levels of abuse and neglect were measured using the Childhood Trauma Questionnaire. Adolescents exposed to high levels of neglect showed reduced RPE-modulated blood oxygenation level dependent response within medial and lateral frontal cortices particularly when exploring novel stimuli (p < 0.05, corrected for multiple comparisons) relative to adolescents exposed to lower levels of neglect. These data expand on previous work by indicating that neglect, but not abuse, is associated with impairments in neural RPE representation within medial and lateral frontal cortices. However, there was no association between neglect and behavioral impairments on the Novelty task, suggesting that these neural differences do not necessarily translate into behavioral differences within the context of the Novelty task.

Symmetric Piecewise Developable Approximations

AbstractWe propose a novel method for generating symmetric piecewise developable approximations for shapes in approximately global reflectional or rotational symmetry. Given a shape and its symmetry constraint, the algorithm contains two crucial steps: (i) a symmetric deformation to achieve a nearly developable model and (ii) a symmetric segmentation aided by the deformed shape. The key to the deformation step is the use of the symmetric implicit neural representations of the shape and the deformation field. A new mesh extraction from the implicit function is introduced to construct a strictly symmetric mesh for the subsequent segmentation. The symmetry constraint is carefully integrated into the partition to achieve the symmetric piecewise developable approximation. We demonstrate the effectiveness of our algorithm over various meshes.

40 Hz sensory stimulation enhances CA3-CA1 coordination and prospective coding during navigation in a mouse model of Alzheimer's disease.

Brain stimulation has emerged as a new potential therapeutic approach to potentially correct or improve altered neural activity in Alzheimer's disease. One such approach, 40 Hz sensory stimulation, or flicker, has been shown to improve cognition in disease models. However, it is not clear how 40 Hz flicker affects neural activity underlying memory processes. Here, we investigate how 40 Hz flicker exposure affects neural activity patterns that are crucial for memory. We find 40Hz flicker increases neural coordination in memory circuits, indicating better communication. Furthermore, 40Hz flicker increased neural representations of future positions, patterns theorized to support memory-based planning. These results indicate that 40 Hz flicker increases key neural activity that is important for memory.

Neural Radiance Fields for Fisheye Driving Scenes Using Edge-Aware Integrated Depth Supervision

Neural radiance fields (NeRF) have become an effective method for encoding scenes into neural representations, allowing for the synthesis of photorealistic views of unseen views from given input images. However, the applicability of traditional NeRF is significantly limited by its assumption that images are captured for object-centric scenes with a pinhole camera. Expanding these boundaries, we focus on driving scenarios using a fisheye camera, which offers the advantage of capturing visual information from a wide field of view. To address the challenges due to the unbounded and distorted characteristics of fisheye images, we propose an edge-aware integration loss function. This approach leverages sparse LiDAR projections and dense depth maps estimated from a learning-based depth model. The proposed algorithm assigns larger weights to neighboring points that have depth values similar to the sensor data. Experiments were conducted on the KITTI-360 and JBNU-Depth360 datasets, which are public and real-world datasets of driving scenarios using fisheye cameras. Experimental results demonstrated that the proposed method is effective in synthesizing novel view images, outperforming existing approaches.

Social conformity updates the neural representation of facial attractiveness

People readily change their behavior to comply with others. However, to which extent they will internalize the social influence remains elusive. In this preregistered electroencephalogram (EEG) study, we investigated how learning from one’s in-group or out-group members about facial attractiveness would change explicit attractiveness ratings and spontaneous neural representations of facial attractiveness. Specifically, we quantified the neural representational similarities of learned faces with prototypical attractive faces during a face perception task without overt social influence and intentional evaluation. We found that participants changed their explicit attractiveness ratings to both in-group and out-group influences. Moreover, social conformity updated spontaneous neural representation of facial attractiveness, an effect particularly evident when participants learned from their in-group members and among those who perceived tighter social norms. These findings offer insights into how group affiliations and individual differences in perceived social norms modulate the internalization of social influence.

Prior cocaine use diminishes encoding of latent information by orbitofrontal, but not medial, prefrontal ensembles.

Maladaptive decision-making is a hallmark of substance use disorders (SUDs), though how drugs of abuse alter neural representations supporting adaptive behavior remains poorly understood. Past studies show that the orbitofrontal (OFC) and prelimbic (PL) cortices are important for decision-making, tracking both task-relevant and latent information. However, previous studies have focused on how drugs of abuse impact the firing rates of individual units. More work at the ensemble level is necessary to accurately characterize potential drug-induced changes. Using single-unit recordings in rats during a multidimensional decision-making task and then applying population- and ensemble-level analyses, we show that prior use of cocaine altered the strength and structure of task-relevant and latent representations in the OFC, changes relatable to suboptimal decision-making in this and perhaps other settings. These data expand our understanding of the neuropathological underpinnings of maladaptive decision-making in SUDs, potentially enabling enhanced future treatment strategies.

Decoding reveals the neural representation of perceived and imagined musical sounds.

Vividly imagining a song or a melody is a skill that many people accomplish with relatively little effort. However, we are only beginning to understand how the brain represents, holds, and manipulates these musical "thoughts." Here, we decoded perceived and imagined melodies from magnetoencephalography (MEG) brain data (N = 71) to characterize their neural representation. We found that, during perception, auditory regions represent the sensory properties of individual sounds. In contrast, a widespread network including fronto-parietal cortex, hippocampus, basal nuclei, and sensorimotor regions hold the melody as an abstract unit during both perception and imagination. Furthermore, the mental manipulation of a melody systematically changes its neural representation, reflecting volitional control of auditory images. Our work sheds light on the nature and dynamics of auditory representations, informing future research on neural decoding of auditory imagination.

Inducing representational change in the hippocampus through real-time neurofeedback.

When you perceive or remember something, other related things come to mind, affecting how these competing items are subsequently perceived and remembered. Such behavioural consequences are believed to result from changes in the overlap of neural representations of these items, especially in the hippocampus. According to multiple theories, hippocampal overlap should increase (integration) when there is high coactivation between cortical representations. However, prior studies used indirect proxies for coactivation by manipulating stimulus similarity or task demands. Here, we induce coactivation in visual cortex more directly using closed-loop neurofeedback from real-time functional magnetic resonance imaging (fMRI). While viewing one object, participants were rewarded for activating the representation of another object as strongly as possible. Across multiple real-time fMRI sessions, participants succeeded in using this neurofeedback to increase coactivation. Compared with a baseline of untrained objects, this protocol led to memory integration in behaviour and the brain: the trained objects became harder for participants to discriminate behaviourally in a categorical perception task and harder to discriminate neurally from patterns of fMRI activity in their hippocampus as a result of losing unique features. These findings demonstrate that neurofeedback can be used to alter and combine memories.This article is part of the theme issue 'Neurofeedback: new territories and neurocognitive mechanisms of endogenous neuromodulation'.