Neural Representation Research Articles

Automatic monitoring of neural activity with single-cell resolution in behaving Hydra

The ability to record every spike from every neuron in a behaving animal is one of the holy grails of neuroscience. Here, we report coming one step closer towards this goal with the development of an end-to-end pipeline that automatically tracks and extracts calcium signals from individual neurons in the cnidarian Hydra vulgaris. We imaged dually labeled (nuclear tdTomato and cytoplasmic GCaMP7s) transgenic Hydra and developed an open-source Python platform (TraSE-IN) for the Tracking and Spike Estimation of Individual Neurons in the animal during behavior. The TraSE-IN platform comprises a series of modules that segments and tracks each nucleus over time and extracts the corresponding calcium activity in the GCaMP channel. Another series of signal processing modules allows robust prediction of individual spikes from each neuron’s calcium signal. This complete pipeline will facilitate the automatic generation and analysis of large-scale datasets of single-cell resolution neural activity in Hydra, and potentially other model organisms, paving the way towards deciphering the neural code of an entire animal.

Acute Aromatase Inhibition Impairs Neural and Behavioral Auditory Scene Analysis in Zebra Finches.

Auditory perception can be significantly disrupted by noise. To discriminate sounds from noise, auditory scene analysis (ASA) extracts the functionally relevant sounds from acoustic input. The zebra finch communicates in noisy environments. Neurons in their secondary auditory pallial cortex (caudomedial nidopallium, NCM) can encode song from background chorus, or scenes, and this capacity may aid behavioral ASA. Furthermore, song processing is modulated by the rapid synthesis of neuroestrogens when hearing conspecific song. To examine whether neuroestrogens support neural and behavioral ASA in both sexes, we retrodialyzed fadrozole (aromatase inhibitor, FAD) and recorded in vivo awake extracellular NCM responses to songs and scenes. We found that FAD affected neural encoding of songs by decreasing responsiveness and timing reliability in inhibitory (narrow-spiking), but not in excitatory (broad-spiking) neurons. Congruently, FAD decreased neural encoding of songs in scenes for both cell types, particularly in females. Behaviorally, we trained birds using operant conditioning and tested their ability to detect songs in scenes after administering FAD orally or injected bilaterally into NCM. Oral FAD increased response bias and decreased correct rejections in females, but not in males. FAD in NCM did not affect performance. Thus, FAD in the NCM impaired neuronal ASA but that did not lead to behavioral disruption suggesting the existence of resilience or compensatory responses. Moreover, impaired performance after systemic FAD suggests involvement of other aromatase-rich networks outside the auditory pathway in ASA. This work highlights how transient estrogen synthesis disruption can modulate higher-order processing in an animal model of vocal communication.

NeuralVDB: High-resolution Sparse Volume Representation using Hierarchical Neural Networks

We introduce NeuralVDB, which improves on an existing industry standard for efficient storage of sparse volumetric data, denoted VDB [Museth 2013 ], by leveraging recent advancements in machine learning. Our novel hybrid data structure can reduce the memory footprints of VDB volumes by orders of magnitude, while maintaining its flexibility and only incurring small (user-controlled) compression errors. Specifically, NeuralVDB replaces the lower nodes of a shallow and wide VDB tree structure with multiple hierarchical neural networks that separately encode topology and value information by means of neural classifiers and regressors respectively. This approach is proven to maximize the compression ratio while maintaining the spatial adaptivity offered by the higher-level VDB data structure. For sparse signed distance fields and density volumes, we have observed compression ratios on the order of 10× to more than 100× from already compressed VDB inputs, with little to no visual artifacts. Furthermore, NeuralVDB is shown to offer more effective compression performance compared to other neural representations such as Neural Geometric Level of Detail [Takikawa et al. 2021 ], Variable Bitrate Neural Fields [Takikawa et al. 2022a ], and Instant Neural Graphics Primitives [Müller et al. 2022 ]. Finally, we demonstrate how warm-starting from previous frames can accelerate training, i.e., compression, of animated volumes as well as improve temporal coherency of model inference, i.e., decompression.

Early visual cortices reveal interrelated item and category representations in aging.

Neural dedifferentiation, the finding that neural representations tend to be less distinct in older adults compared with younger adults, has been associated with age-related declines in memory performance. Most studies assessing the relation between memory and neural dedifferentiation have evaluated how age impacts the distinctiveness of neural representations for different visual categories (e.g., scenes and objects). However, how age impacts the quality of neural representations at the level of individual items is still an open question. Here, we present data from an age-comparative fMRI study that aimed to understand how the distinctiveness of neural representations for individual stimuli differs between younger and older adults and relates to memory outcomes. Pattern similarity searchlight analyses yielded indicators of neural dedifferentiation at the level of individual items as well as at the category level in posterior occipital cortices. We asked whether age differences in neural distinctiveness at each representational level were associated with inter- and/or intraindividual variability in memory performance. While age-related dedifferentiation at both the item and category level related to between-person differences in memory, neural distinctiveness at the category level also tracked within-person variability in memory performance. Concurrently, neural distinctiveness at the item level was strongly associated with neural distinctiveness at the category level both within and across participants, elucidating a potential representational mechanism linking item- and category-level distinctiveness. In sum, we provide evidence that age-related neural dedifferentiation co-exists across multiple representational levels and is related to memory performance.Significance Statement Age-related memory decline has been associated with neural dedifferentiation, the finding that older adults have less distinctive neural representations than younger adults. This has been mostly shown for category information, while evidence for age differences in the specificity of item representations is meager. We used pattern similarity searchlight analyses to find indicators of neural dedifferentiation at both levels of representation (category and item) and linked distinctiveness to memory performance. Both item- and category-level dedifferentiation in the calcarine cortex were related to interindividual differences in memory performance, while category-level distinctiveness further tracked intraindividual variability. Crucially, neural distinctiveness was strongly tied between the item and category levels, indicating that intersecting representational properties of posterior occipital cortices reflect both individual exemplars and categories.

Investigation of a neural implicit representation tomography method for flow diagnostics

Abstract In this work, a new gridless approach to tomographic reconstruction of 3D flow fields is introduced and investigated. The approach, termed here as FluidNeRF, is based on the concept of volume representation through Neural Radiance Fields (NeRF). NeRF represents a 3D volume as a continuous function using a deep neural network. In FluidNeRF, the neural network is a function of 3D spatial coordinates in the volume and produces an intensity of light per unit volume at that position. The network is trained using the loss between measured and rendered 2D projections similar to other multi-camera tomography techniques. Projections are rendered using an emission-based integrated line-of-sight method where light rays are traced through the volume; the network is used to determine intensity values along the ray. This paper investigates the influence of the NeRF hyperparameters, camera layout and spacing, and image noise on the reconstruction quality as well as the computational cost. A DNS-generated synthetic turbulent jet is used as a ground-truth representative flow field. Results obtained with FluidNeRF are compared to an adaptive simultaneous algebraic reconstruction technique (ASART), which is representative of a conventional reconstruction technique. Results show that FluidNeRF matches or outperforms ASART in reconstruction quality, is more robust to noise, and offers several advantages that make it more flexible and thus suitable for extension to other flow measurement techniques and scaling to larger-scale problems.

Dissimilarities of neural representations of extinction trials are associated with extinction learning performance and renewal level.

Renewal of extinguished responses is associated with higher activity in specific extinction-relevant brain regions, i.e., hippocampus (HC), inferior frontal gyrus (IFG), and ventromedial PFC (vmPFC). HC is involved in processing of context information, while IFG and vmPFC use such context information for selecting and deciding among competing response options. However, it is as yet unknown to what extent trials with changed versus unchanged outcome, or extinction trials that evoke renewal (i.e., extinction context differs from acquisition and test context: ABA trials) and trials that do not (i.e., same context in all phases: AAA trials) are represented differentially in extinction-relevant brain regions. In this study, we applied representational similarity analysis (RSA) to determine differences in neural representations of these trial types and their relationship to extinction error rates and renewal level. Overall, individuals with renewal (REN) and those without (NoREN) did not differ significantly in their discrimination levels between ABA and AAA extinction trials, with the exception of right posterior HC, where REN exhibited more pronounced context-related discrimination. In addition, higher dissimilarity of representations in bilateral posterior HC, as well as in several IFG regions, during extinction learning was linked to lower ABA renewal rates. Both REN and NoREN benefitted from prediction error feedback from ABA extinction errors for context- and outcome-related discrimination of trials in IFG, vmPFC, and HC, but only the NoREN group also benefitted from error feedback from AAA extinction errors. Thus, while in both groups the presence of a novel context supported formation of distinct representations, only in NoREN the expectancy violation of the surprising change of outcome alone had a similar effect. In addition, only in NoREN context-related discrimination was linked to error feedback in vmPFC. In summary, the findings show that context- and outcome-related discrimination of trials in HC, vmPFC, and IFG is linked to extinction learning errors, regardless of renewal propensity, and at the same time point towards differential context processing strategies in REN and NoREN. Moreover, better discrimination of context-related trials during extinction learning promotes less renewal during extinction recall, suggesting that renewal may be related to suboptimal context-related trial discrimination.

Is attention necessary for the representational advantage of good exemplars over bad exemplars?

Real-world (rw-) statistical regularities, or expectations about the visual world learned over a lifetime, have been found to be associated with scene perception efficiency. For example, good (i.e., highly representative) exemplars of basic scene categories, one example of an rw-statistical regularity, are detected more readily than bad exemplars of the category. Similarly, good exemplars achieve higher multivariate pattern analysis (MVPA) classification accuracy than bad exemplars in scene-responsive regions of interest, particularly in the parahippocampal place area (PPA). However, it is unclear whether the good exemplar advantages observed depend on or are even confounded by selective attention. Here, we ask whether the observed neural advantage of the good scene exemplars requires full attention. We used a dual-task paradigm to manipulate attention and exemplar representativeness while recording neural responses with functional magnetic resonance imaging (fMRI). Both univariate analysis and MVPA were adopted to examine the effect of representativeness. In the attend-to-scenes condition, our results replicated an earlier study showing that good exemplars evoke less activity but a clearer category representation than bad exemplars. Importantly, similar advantages of the good exemplars were also observed when participants were distracted by a serial visual search task demanding a high attention load. In addition, cross-decoding between attended and distracted representations revealed that attention resulted in a quantitative (increased activation) rather than qualitative (altered activity patterns) improvement of the category representation, particularly for good exemplars. We, therefore, conclude that the effect of category representativeness on neural representations does not require full attention.

Induced neural phase precession through exogenous electric fields

The gradual shifting of preferred neural spiking relative to local field potentials (LFPs), known as phase precession, plays a prominent role in neural coding. Correlations between the phase precession and behavior have been observed throughout various brain regions. As such, phase precession is suggested to be a global neural mechanism that promotes local neuroplasticity. However, causal evidence and neuroplastic mechanisms of phase precession are lacking so far. Here we show a causal link between LFP dynamics and phase precession. In three experiments, we modulated LFPs in humans, a non-human primate, and computational models using alternating current stimulation. We show that continuous stimulation of motor cortex oscillations in humans lead to a gradual phase shift of maximal corticospinal excitability by ~90°. Further, exogenous alternating current stimulation induced phase precession in a subset of entrained neurons (~30%) in the non-human primate. Multiscale modeling of realistic neural circuits suggests that alternating current stimulation-induced phase precession is driven by NMDA-mediated synaptic plasticity. Altogether, the three experiments provide mechanistic and causal evidence for phase precession as a global neocortical process. Alternating current-induced phase precession and consequently synaptic plasticity is crucial for the development of novel therapeutic neuromodulation methods.

Fast Digital Orthophoto Generation: A Comparative Study of Explicit and Implicit Methods

A digital orthophoto is an image with geometric accuracy and no distortion. It is acquired through a top view of the scene and finds widespread applications in map creation, planning, and related fields. This paper classifies the algorithms for digital orthophoto generation into two groups: explicit methods and implicit methods. Explicit methods rely on traditional geometric methods, obtaining geometric structure presented with explicit parameters with Multi-View Stereo (MVS) theories, as seen in our proposed Top view constrained Dense Matching (TDM). Implicit methods rely on neural rendering, obtaining implicit neural representation of scenes through the training of neural networks, as exemplified by Neural Radiance Fields (NeRFs). Both of them obtain digital orthophotos via rendering from a top-view perspective. In addition, this paper conducts an in-depth comparative study between explicit and implicit methods. The experiments demonstrate that both algorithms meet the measurement accuracy requirements and exhibit a similar level of quality in terms of generated results. Importantly, the explicit method shows a significant advantage in terms of efficiency, with a time consumption reduction of two orders of magnitude under our latest Compute Unified Device Architecture (CUDA) version TDM algorithm. Although explicit and implicit methods differ significantly in their representation forms, they share commonalities in the implementation across algorithmic stages. These findings highlight the potential advantages of explicit methods in orthophoto generation while also providing beneficial references and practical guidance for fast digital orthophoto generation using implicit methods.

Midbrain signaling of identity prediction errors depends on orbitofrontal cortex networks

Outcome-guided behavior requires knowledge about the identity of future rewards. Previous work across species has shown that the dopaminergic midbrain responds to violations in expected reward identity and that the lateral orbitofrontal cortex (OFC) represents reward identity expectations. Here we used network-targeted transcranial magnetic stimulation (TMS) and functional magnetic resonance imaging (fMRI) during a trans-reinforcer reversal learning task to test the hypothesis that outcome expectations in the lateral OFC contribute to the computation of identity prediction errors (iPE) in the midbrain. Network-targeted TMS aiming at lateral OFC reduced the global connectedness of the lateral OFC and impaired reward identity learning in the first block of trials. Critically, TMS disrupted neural representations of expected reward identity in the OFC and modulated iPE responses in the midbrain. These results support the idea that iPE signals in the dopaminergic midbrain are computed based on outcome expectations represented in the lateral OFC.

Learning attentional templates for value-based decision-making

Attention filters sensory inputs to enhance task-relevant information. It is guided by an "attentional template" that represents the stimulus features that are currently relevant. To understand how the brain learns and uses templates, we trained monkeys to perform a visual search task that required them to repeatedly learn new attentional templates. Neural recordings found that templates were represented across the prefrontal and parietal cortex in a structured manner, such that perceptually neighboring templates had similar neural representations. When the task changed, a new attentional template was learned by incrementally shifting the template toward rewarded features. Finally, we found that attentional templates transformed stimulus features into a common value representation that allowed the same decision-making mechanisms to deploy attention, regardless of the identity of the template. Altogether, our results provide insight into the neural mechanisms by which the brain learns to control attention and how attention can be flexibly deployed acrosstasks.

Mnemonic But Not Contextual Feedback Signals Defy Dedifferentiation in the Aging Early Visual Cortex.

Perception is an intricate interplay between feedforward visual input and internally generated feedback signals that comprise concurrent contextual and time-distant mnemonic (episodic and semantic) information. Yet, an unresolved question is how the composition of feedback signals changes across the lifespan and to what extent feedback signals undergo age-related dedifferentiation, that is, a decline in neural specificity. Previous research on this topic has focused on feedforward perceptual representation and episodic memory reinstatement, suggesting reduced fidelity of neural representations at the item and category levels. In this fMRI study, we combined an occlusion paradigm that filters feedforward input to the visual cortex and multivariate analysis techniques to investigate the information content in cortical feedback, focusing on age-related differences in its composition. We further asked to what extent differentiation in feedback signals (in the occluded region) is correlated to differentiation in feedforward signals. Comparing younger (18-30 years) and older female and male adults (65-75 years), we found that contextual but not mnemonic feedback was prone to age-related dedifferentiation. Semantic feedback signals were even better differentiated in older adults, highlighting the growing importance of generalized knowledge across ages. We also found that differentiation in feedforward signals was correlated with differentiation in episodic but not semantic feedback signals. Our results provide evidence for age-related adjustments in the composition of feedback signals and underscore the importance of examining dedifferentiation in aging for both feedforward and feedback processing.

Convolutional neural networks reveal properties of reach-to-grasp encoding in posterior parietal cortex

Deep neural networks (DNNs) are widely adopted to decode motor states from both non-invasively and invasively recorded neural signals, e.g., for realizing brain-computer interfaces. However, the neurophysiological interpretation of how DNNs make the decision based on the input neural activity is limitedly addressed, especially when applied to invasively recorded data. This reduces decoder reliability and transparency, and prevents the exploitation of decoders to better comprehend motor neural encoding. Here, we adopted an explainable artificial intelligence approach – based on a convolutional neural network and an explanation technique – to reveal spatial and temporal neural properties of reach-to-grasping from single-neuron recordings of the posterior parietal area V6A. The network was able to accurately decode 5 different grip types, and the explanation technique automatically identified the cells and temporal samples that most influenced the network prediction. Grip encoding in V6A neurons already started at movement preparation, peaking during movement execution. A difference was found within V6A: dorsal V6A neurons progressively encoded more for increasingly advanced grips, while ventral V6A neurons for increasingly rudimentary grips, with both subareas following a linear trend between the amount of grip encoding and the level of grip skills. By revealing the elements of the neural activity most relevant for each grip with no a priori assumptions, our approach supports and advances current knowledge about reach-to-grasp encoding in V6A, and it may represent a general tool able to investigate neural correlates of motor or cognitive tasks (e.g., attention and memory tasks) from single-neuron recordings.

Uncertainty Quantification of Neural Reflectance Fields for Underwater Scenes

Neural radiance fields and neural reflectance fields are novel deep learning methods for generating novel views of 3D scenes from 2D images. To extend the neural scene representation techniques to complex underwater environments, beyond neural reflectance fields underwater (BNU) was proposed, which considers the relighting conditions of on-aboard light sources by using neural reflectance fields, and approximates the attenuation and backscatter effects of water with an additional constant. Because the quality of the neural representation of underwater scenes is critical to downstream tasks such as marine surveying and mapping, the model reliability should be considered and evaluated. However, current neural reflectance models lack the ability of quantifying the uncertainty of underwater scenes that are not directly observed during training, which hinders their widespread use in the field of underwater unmanned autonomous navigation. To address this issue, we introduce an ensemble strategy to BNU that quantifies cognitive uncertainty in color space and unobserved regions with the expectation and variance of RGB values and termination probabilities along the ray. We also employ a regularization method to smooth the density of the underwater neural reflectance model. The effectiveness of the present method is demonstrated in numerical experiments.

Transfer learning graph representations of molecules for pKa, 13C-NMR, and solubility

We explore transfer learning models from a pre-trained graph convolutional neural network representation of molecules, obtained from SchNet, to predict 13C-NMR, pKa, and log S solubility. SchNet learns a graph representation of a molecule by associating each atom with an “embedding vector” and interacts the atom-embeddings with each other by leveraging graph convolutional filters on their interatomic distances. We pre-trained SchNet on molecular energy and demonstrate that the pre-trained atomistic embeddings can then be used as a transferable representation for a wide array of properties. On the one hand, for atomic properties such as micro-pK1 and 13C-NMR, we investigate two models, one linear and one neural net, that input pre-trained atom-embeddings of a particular atom (e.g. carbon) and predict a local property (e.g., 13C-NMR). On the other hand, for molecular properties such as solubility, a size-extensive graph model is built using the embeddings of all atoms in the molecule as input. For all cases, qualitatively correct predictions are made with relatively little training data (<1000 training points), showcasing the ease with which pre-trained embeddings pick up on important chemical patterns. The proposed models successfully capture well-understood trends of pK1 and solubility. This study advances our understanding of current neural net graph representations and their capacity for transfer learning applications in chemistry.

Neural correlates of willingness to pay for items: A meta-analysis of functional magnetic resonance imaging studies

Willingness to pay (WTP) pervades every marketplace transaction, therefore, understanding how the brain makes bidding decisions is essential in consumer neuroscience. Although some neuroimaging studies have investigated the neural networks of WTP, systematic understanding remains limited. This study identified reliable neural networks activated by the WTP across different reward types and assessed common and distinct neural networks for different reward types (food and other) bids. We conducted an activation likelihood estimation (ALE) meta-analysis on WTP across different reward types (25 studies; 254 foci; 705 participants), and to compared neural representations of WTP for food reward (22 studies; 232 foci; 628 participants) and other rewards (7 studies, 61 foci; 177 participants). The ALE results revealed that the brain centers of WTP for different rewards mainly consist of the bilateral inferior frontal gyrus (IFG), bilateral insula, bilateral anterior cingulate cortex (ACC), along with the left caudate. This suggests that neural networks encoding WTP for different rewards consist of brain regions associated with reward processing, cost-benefit calculations, and goal-directed action activities. In addition, consistent activation of the bilateral IFG and bilateral insula for food but no other rewards bids suggest their involvement in the neural network of appetite. WTP for food and other rewards commonly activated ACC, suggesting a common region encoding bids for different rewards. Our findings provide novel insights into neural networks associated with WTP for food and other rewards bids and the mechanisms underlying WTP across different reward types.

Intracranial electroencephalography reveals effector-independent evidence accumulation dynamics in multiple human brain regions.

Neural representations of perceptual decision formation that are abstracted from specific motor requirements have previously been identified in humans using non-invasive electrophysiology; however, it is currently unclear where these originate in the brain. Here we capitalized on the high spatiotemporal precision of intracranial EEG to localize such abstract decision signals. Participants undergoing invasive electrophysiological monitoring for epilepsy were asked to judge the direction of random-dot stimuli and respond either with a speeded button press (N = 24), or vocally, after a randomized delay (N = 12). We found a widely distributed motor-independent network of regions where high-frequency activity exhibited key characteristics consistent with evidence accumulation, including a gradual buildup that was modulated by the strength of the sensory evidence, and an amplitude that predicted participants' choice accuracy and response time. Our findings offer a new view on the brain networks governing human decision-making.

EDiGS : Extended Divergence guided shape implicit neural representation for unoriented point clouds

eDiGS : Extended Divergence guided shape implicit neural representation for unoriented point clouds

Encoding of melody in the human auditory cortex.

Melody is a core component of music in which discrete pitches are serially arranged to convey emotion and meaning. Perception varies along several pitch-based dimensions: (i) the absolute pitch of notes, (ii) the difference in pitch between successive notes, and (iii) the statistical expectation of each note given prior context. How the brain represents these dimensions and whether their encoding is specialized for music remains unknown. We recorded high-density neurophysiological activity directly from the human auditory cortex while participants listened to Western musical phrases. Pitch, pitch-change, and expectation were selectively encoded at different cortical sites, indicating a spatial map for representing distinct melodic dimensions. The same participants listened to spoken English, and we compared responses to music and speech. Cortical sites selective for music encoded expectation, while sites that encoded pitch and pitch-change in music used the same neural code to represent equivalent properties of speech. Findings reveal how the perception of melody recruits both music-specific and general-purpose sound representations.

Sequential neuronal processing of number values, abstract decision, and action in the primate prefrontal cortex.

Decision-making requires processing of sensory information, comparing the gathered evidence to make a judgment, and performing the action to communicate it. How neuronal representations transform during this cascade of representations remains a matter of debate. Here, we studied the succession of neuronal representations in the primate prefrontal cortex (PFC). We trained monkeys to judge whether a pair of sequentially presented displays had the same number of items. We used a combination of single neuron and population-level analyses and discovered a sequential transformation of represented information with trial progression. While numerical values were initially represented with high precision and in conjunction with detailed information such as order, the decision was encoded in a low-dimensional subspace of neural activity. This decision encoding was invariant to both retrospective numerical values and prospective motor plans, representing only the binary judgment of "same number" versus "different number," thus facilitating the generalization of decisions to novel number pairs. We conclude that this transformation of neuronal codes within the prefrontal cortex supports cognitive flexibility and generalizability of decisions to new conditions.