Representation Of Stimuli Research Articles

Experience-driven development of decision-related representations in the auditory cortex.

Associating sensory stimuli with behavioral significance induces substantial changes in stimulus representations. Recent studies suggest that primary sensory cortices not only adjust representations of task-relevant stimuli, but actively participate in encoding features of the decision-making process. We sought to determine whether this trait is innate in sensory cortices or if choice representation develops with time and experience. To trace choice representation development, we perform chronic two-photon calcium imaging in the primary auditory cortex of head-fixed mice while they gain experience in a tone detection task with a delayed decision window. Our results reveal a progressive increase in choice-dependent activity within a specific subpopulation of neurons, aligning with growing task familiarity and adapting to changing task rules. Furthermore, task experience correlates with heightened synchronized activity in these populations and the ability to differentiate between different types of behavioral decisions. Notably, the activity of this subpopulation accurately decodes the same action at different task phases. Our findings establish a dynamic restructuring of population activity in the auditory cortex to encode features of the decision-making process that develop over time and refines with experience.

Valence and Salience Encoding in the Central Amygdala.

The central amygdala (CeA) has emerged as an important brain region for regulating both negative (fear and anxiety) and positive (reward) affective behaviors. The CeA has been proposed to encode affective information in the form of valence (whether the stimulus is good or bad) or salience (how significant is the stimulus), but the extent to which these two types of stimulus representation occur in the CeA is not known. Here, we used single cell calcium imaging in mice during appetitive and aversive conditioning and found that majority of CeA neurons (∼65%) encode the valence of the unconditioned stimulus (US) with a smaller subset of cells (∼15%) encoding the salience of the US. Valence and salience encoding of the conditioned stimulus (CS) was also observed, albeit to a lesser extent. These findings show that the CeA is a site of convergence for encoding oppositely valenced US information.

Proximity within adolescent peer networks predicts neural similarity during affective experience.

Individuals befriend others who are similar to them. One important source of similarity in relationships is similarity in felt emotion. In the present study, we used novel methods to assess whether greater similarity in the multivoxel brain representation of affective stimuli was associated with adolescents' proximity within real-world school-based social networks. We examined dyad-level neural similarity within a set of brain regions associated with the representation of affect including the ventromedial prefrontal cortex (vmPFC), amygdala, insula, and temporal pole. Greater proximity was associated with greater vmPFC neural similarity during pleasant and neutral experiences. Moreover, we used unsupervised clustering on social networks to identify groups of friends and observed that individuals from the same (versus different) friend groups were more likely to have greater vmPFC neural similarity during pleasant and negative experiences. These findings suggest that similarity in the multivoxel brain representation of affect may play an important role in adolescent friendships.

Sensory-memory interactions via modular structure explain errors in visual working memory.

Errors in stimulus estimation reveal how stimulus representation changes during cognitive processes. Repulsive bias and minimum variance observed near cardinal axes are well-known error patterns typically associated with visual orientation perception. Recent experiments suggest that these errors continuously evolve during working memory, posing a challenge that neither static sensory models nor traditional memory models can address. Here, we demonstrate that these evolving errors, maintaining characteristic shapes, require network interaction between two distinct modules. Each module fulfills efficient sensory encoding and memory maintenance, which cannot be achieved simultaneously in a single-module network. The sensory module exhibits heterogeneous tuning with strong inhibitory modulation reflecting natural orientation statistics. While the memory module, operating alone, supports homogeneous representation via continuous attractor dynamics, the fully connected network forms discrete attractors with moderate drift speed and nonuniform diffusion processes. Together, our work underscores the significance of sensory-memory interaction in continuously shaping stimulus representation during working memory.

Spatial Predictive Context Speeds Up Visual Search by Biasing Local Attentional Competition.

The human visual system is equipped to rapidly and implicitly learn and exploit the statistical regularities in our environment. Within visual search, contextual cueing demonstrates how implicit knowledge of scenes can improve search performance. This is commonly interpreted as spatial context in the scenes becoming predictive of the target location, which leads to a more efficient guidance of attention during search. However, what drives this enhanced guidance is unknown. First, it is under debate whether the entire scene (global context) or more local context drives this phenomenon. Second, it is unclear how exactly improved attentional guidance is enabled by target enhancement and distractor suppression. In the present magnetoencephalography experiment, we leveraged rapid invisible frequency tagging to answer these two outstanding questions. We found that the improved performance when searching implicitly familiar scenes was accompanied by a stronger neural representation of the target stimulus, at the cost specifically of those distractors directly surrounding the target. Crucially, this biasing of local attentional competition was behaviorally relevant when searching familiar scenes. Taken together, we conclude that implicitly learned spatial predictive context improves how we search our environment by sharpening the attentional field.

What Is Faster than Where in Vocal Emotional Perception.

Voices carry a vast amount of information about speakers (e.g., emotional state; spatial location). Neuroimaging studies postulate that spatial ("where") and emotional ("what") cues are processed by partially independent processing streams. Although behavioral evidence reveals interactions between emotion and space, the temporal dynamics of these processes in the brain and its modulation by attention remain unknown. We investigated whether and how spatial and emotional features interact during voice processing as a function of attention focus. Spatialized nonverbal vocalizations differing in valence (neutral, amusement, anger) were presented at different locations around the head, whereas listeners discriminated either the spatial location or emotional quality of the voice. Neural activity was measured with ERPs of the EEG. Affective ratings were collected at the end of the EEG session. Emotional vocalizations elicited decreased N1 but increased P2 and late positive potential amplitudes. Interactions of space and emotion occurred at the salience detection stage: neutral vocalizations presented at right (vs. left) locations elicited increased P2 amplitudes, but no such differences were observed for emotional vocalizations. When task instructions involved emotion categorization, the P2 was increased for vocalizations presented at front (vs. back) locations. Behaviorally, only valence and arousal ratings showed emotion-space interactions. These findings suggest that emotional representations are activated earlier than spatial representations in voice processing. The perceptual prioritization of emotional cues occurred irrespective of task instructions but was not paralleled by an augmented stimulus representation in space. These findings support the differential responding to emotional information by auditory processing pathways.

Maintenance and transformation of representational formats during working memory prioritization

Visual working memory depends on both material-specific brain areas in the ventral visual stream (VVS) that support the maintenance of stimulus representations and on regions in the prefrontal cortex (PFC) that control these representations. How executive control prioritizes working memory contents and whether this affects their representational formats remains an open question, however. Here, we analyzed intracranial EEG (iEEG) recordings in epilepsy patients with electrodes in VVS and PFC who performed a multi-item working memory task involving a retro-cue. We employed Representational Similarity Analysis (RSA) with various Deep Neural Network (DNN) architectures to investigate the representational format of prioritized VWM content. While recurrent DNN representations matched PFC representations in the beta band (15–29 Hz) following the retro-cue, they corresponded to VVS representations in a lower frequency range (3–14 Hz) towards the end of the maintenance period. Our findings highlight the distinct coding schemes and representational formats of prioritized content in VVS and PFC.

Contrast gain control is a reparameterization of a population response curve.

Neurons in primary visual cortex (area V1) adapt in varying degrees to the average contrast of the environment, suggesting that the representation of visual stimuli may interact with the state of cortical gain control in complex ways. To investigate this possibility, we measured and analyzed the responses of neural populations in mouse V1 to visual stimuli as a function of contrast in different environments, each characterized by a unique distribution of contrast values. Our findings reveal that, for a fixed stimulus, the population response can be described by a vector function r(gec), where the gain ge is a decreasing function of the mean contrast of the environment. Thus, gain control can be viewed as a reparameterization of a population response curve, which is invariant across environments. Different stimuli are mapped to distinct curves, all originating from a common origin, corresponding to a zero-contrast response. Altogether, our findings provide a straightforward, geometric interpretation of contrast gain control at the population level and show that changes in gain are well matched among members of a population.NEW & NOTEWORTHY The authors study the responses of neural populations in mouse primary visual cortex as a function of stimulus contrast. Measurements are performed in different "environments," each characterized by a different distribution of contrast values. They find that responses across environments can be viewed as a reparameterization of a single contrast-response curve, offering a simple, geometric account of contrast gain control in neural populations.

Neural Activity Differentiates Novel and Learned Event Boundaries.

People parse continuous experiences at natural breakpoints called event boundaries, which is important for understanding an environment's causal structure and for responding to uncertainty within it. However, it remains unclear how different forms of uncertainty affect the parsing of continuous experiences and how such uncertainty influences the brain's processing of ongoing events. We exposed human participants of both sexes (N = 34) to a continuous sequence of semantically meaningless images. We generated sequences from random walks through a graph that grouped images into temporal communities. After learning, we asked participants to segment another sequence at natural breakpoints (event boundaries). Participants segmented the sequence at learned transitions between communities, as well as at novel transitions, suggesting that people can segment temporally extended experiences into events based on learned structure as well as prediction error. Greater segmentation at novel boundaries was associated with enhanced parietal scalp electroencephalography (EEG) activity between 250 and 450 ms after the stimulus onset. Multivariate classification of EEG activity showed that novel and learned boundaries evoked distinct patterns of neural activity, particularly theta band power in posterior electrodes. Learning also led to distinct neural representations for stimuli within the temporal communities, while neural activity at learned boundary nodes showed predictive evidence for the adjacent community. The data show that people segment experiences at both learned and novel boundaries and suggest that learned event boundaries trigger retrieval of information about the upcoming community that could underlie anticipation of the next event in a sequence.

Stimulus representations in neural priority maps are equally enhanced by attention independent of the number of attended locations

Stimulus representations in neural priority maps are equally enhanced by attention independent of the number of attended locations

Memory representations during slow change blindness.

Classic change blindness is the phenomenon where seemingly obvious changes that coincide with visual disruptions (such as blinks or brief blanks) go unnoticed by an attentive observer. Some early work into the causes of classic change blindness suggested that any pre-change stimulus representation is overwritten by a representation of the altered post-change stimulus, preventing change detection. However, recent work revealed that, even when observers do maintain memory representations of both the pre- and post-change stimulus states, they can still miss the change, suggesting that change blindness can also arise from a failure to compare the stored representations. Here, we studied slow change blindness, a related phenomenon that occurs even in the absence of visual disruptions when the change occurs sufficiently slowly, to determine whether it could be explained by conclusions from classic change blindness. Across three different slow change blindness experiments we found that observers who consistently failed to notice the change had access to at least two memory representations of the changing display. One representation was precise but short lived: a detailed representation of the more recent stimulus states, but fragile. The other representation lasted longer but was fairly general: stable but too coarse to differentiate the various stages of the change. These findings suggest that, although multiple representations are formed, the failure to compare hypotheses might not explain slow change blindness; even if a comparison were made, the representations would be too sparse (longer term stores) or too fragile (short-lived stores) for such comparison to inform about the change.

Brain structure and activity predicting cognitive maturation in adolescence.

Cognitive abilities of primates, including humans, continue to improve through adolescence 1,2. While a range of changes in brain structure and connectivity have been documented 3,4, how they affect neuronal activity that ultimately determines performance of cognitive functions remains unknown. Here, we conducted a multilevel longitudinal study of monkey adolescent neurocognitive development. The developmental trajectory of neural activity in the prefrontal cortex accounted remarkably well for working memory improvements. While complex aspects of activity changed progressively during adolescence, such as the rotation of stimulus representation in multidimensional neuronal space, which has been implicated in cognitive flexibility, even simpler attributes, such as the baseline firing rate in the period preceding a stimulus appearance had predictive power over behavior. Unexpectedly, decreases in brain volume and thickness, which are widely thought to underlie cognitive changes in humans 5 did not predict well the trajectory of neural activity or cognitive performance changes. Whole brain cortical volume in particular, exhibited an increase and reached a local maximum in late adolescence, at a time of rapid behavioral improvement. Maturation of long-distance white matter tracts linking the frontal lobe with areas of the association cortex and subcortical regions best predicted changes in neuronal activity and behavior. Our results provide evidence that optimization of neural activity depending on widely distributed circuitry effects cognitive development in adolescence.

Interaction of spatial attention and the associated reward value of audiovisual objects

Reward value and selective attention both enhance the representation of sensory stimuli at the earliest stages of processing. It is still debated whether and how reward-driven and attentional mechanisms interact to influence perception. Here we ask whether the interaction between reward value and selective attention depends on the sensory modality through which the reward information is conveyed. Human participants first learned the reward value of uni-modal visual and auditory stimuli during a conditioning phase. Subsequently, they performed a target detection task on bimodal stimuli containing a previously rewarded stimulus in one, both, or neither of the modalities. Additionally, participants were required to focus their attention on one side and only report targets on the attended side. Our results showed a strong modulation of visual and auditory event-related potentials (ERPs) by spatial attention. We found no main effect of reward value but importantly we found an interaction effect as the strength of attentional modulation of the ERPs was significantly affected by the reward value. When reward effects were examined separately with respect to each modality, auditory value-driven modulation of attention was found to dominate the ERP effects whereas visual reward value on its own led to no effect, likely due to its interference with the target processing. These results inspire a two-stage model where first the salience of a high reward stimulus is enhanced on a local priority map specific to each sensory modality, and at a second stage reward value and top-down attentional mechanisms are integrated across sensory modalities to affect perception.

Pupil-Linked Arousal Modulates Precision of Stimulus Representation in Cortex.

Neural responses are naturally variable from one moment to the next, even when the stimulus is held constant. What factors might underlie this variability in neural population activity? We hypothesized that spontaneous fluctuations in cortical stimulus representations are created by changes in arousal state. We tested the hypothesis using a combination of fMRI, probabilistic decoding methods, and pupillometry. Human participants (20 female, 12 male) were presented with gratings of random orientation. Shortly after viewing the grating, participants reported its orientation and gave their level of confidence in this judgment. Using a probabilistic fMRI decoding technique, we quantified the precision of the stimulus representation in the visual cortex on a trial-by-trial basis. Pupil size was recorded and analyzed to index the observer's arousal state. We found that the precision of the cortical stimulus representation, reported confidence, and variability in the behavioral orientation judgments varied from trial to trial. Interestingly, these trial-by-trial changes in cortical and behavioral precision and confidence were linked to pupil size and its temporal rate of change. Specifically, when the cortical stimulus representation was more precise, the pupil dilated more strongly prior to stimulus onset and remained larger during stimulus presentation. Similarly, stronger pupil dilation during stimulus presentation was associated with higher levels of subjective confidence, a secondary measure of sensory precision, as well as improved behavioral performance. Taken together, our findings support the hypothesis that spontaneous fluctuations in arousal state modulate the fidelity of the stimulus representation in the human visual cortex, with clear consequences for behavior.

Functional connectivity across the human subcortical auditory system using an autoregressive matrix-Gaussian copula graphical model approach with partial correlations.

The auditory system comprises multiple subcortical brain structures that process and refine incoming acoustic signals along the primary auditory pathway. Due to technical limitations of imaging small structures deep inside the brain, most of our knowledge of the subcortical auditory system is based on research in animal models using invasive methodologies. Advances in ultrahigh-field functional magnetic resonance imaging (fMRI) acquisition have enabled novel noninvasive investigations of the human auditory subcortex, including fundamental features of auditory representation such as tonotopy and periodotopy. However, functional connectivity across subcortical networks is still underexplored in humans, with ongoing development of related methods. Traditionally, functional connectivity is estimated from fMRI data with full correlation matrices. However, partial correlations reveal the relationship between two regions after removing the effects of all other regions, reflecting more direct connectivity. Partial correlation analysis is particularly promising in the ascending auditory system, where sensory information is passed in an obligatory manner, from nucleus to nucleus up the primary auditory pathway, providing redundant but also increasingly abstract representations of auditory stimuli. While most existing methods for learning conditional dependency structures based on partial correlations assume independently and identically Gaussian distributed data, fMRI data exhibit significant deviations from Gaussianity as well as high-temporal autocorrelation. In this paper, we developed an autoregressive matrix-Gaussian copula graphical model (ARMGCGM) approach to estimate the partial correlations and thereby infer the functional connectivity patterns within the auditory system while appropriately accounting for autocorrelations between successive fMRI scans. Our results show strong positive partial correlations between successive structures in the primary auditory pathway on each side (left and right), including between auditory midbrain and thalamus, and between primary and associative auditory cortex. These results are highly stable when splitting the data in halves according to the acquisition schemes and computing partial correlations separately for each half of the data, as well as across cross-validation folds. In contrast, full correlation-based analysis identified a rich network of interconnectivity that was not specific to adjacent nodes along the pathway. Overall, our results demonstrate that unique functional connectivity patterns along the auditory pathway are recoverable using novel connectivity approaches and that our connectivity methods are reliable across multiple acquisitions.

Desegregation of neuronal predictive processing.

Neural circuits construct internal 'world-models' to guide behavior. The predictive processing framework posits that neural activity signaling sensory predictions and concurrently computing prediction-errors is a signature of those internal models. Here, to understand how the brain generates predictions for complex sensorimotor signals, we investigate the emergence of high-dimensional, multi-modal predictive representations in recurrent networks. We find that robust predictive processing arises in a network with loose excitatory/inhibitory balance. Contrary to previous proposals of functionally specialized cell-types, the network exhibits desegregation of stimulus and prediction-error representations. We confirmed these model predictions by experimentally probing predictive-coding circuits using a rich stimulus-set to violate learned expectations. When constrained by data, our model further reveals and makes concrete testable experimental predictions for the distinct functional roles of excitatory and inhibitory neurons, and of neurons in different layers along a laminar hierarchy, in computing multi-modal predictions. These results together imply that in natural conditions, neural representations of internal models are highly distributed, yet structured to allow flexible readout of behaviorally-relevant information. The generality of our model advances the understanding of computation of internal models across species, by incorporating different types of predictive computations into a unified framework.

Prefrontal neuronal dynamics in the absence of task execution

Prefrontal cortical activity represents stimuli in working memory tasks in a low-dimensional manifold that transforms over the course of a trial. Such transformations reflect specific cognitive operations, so that, for example, the rotation of stimulus representations is thought to reduce interference by distractor stimuli. Here we show that rotations occur in the low-dimensional activity space of prefrontal neurons in naïve male monkeys (Macaca mulatta), while passively viewing familiar stimuli. Moreover, some aspects of these rotations remain remarkably unchanged after training to perform working memory tasks. Significant training effects are still present in population dynamics, which further distinguish correct and error trials during task execution. Our results reveal automatic functions of prefrontal neural circuits allow transformations that may aid cognitive flexibility.

Selective Attention and Decision-Making Have Separable Neural Bases in Space and Time.

Attention and decision-making processes are fundamental to cognition. However, they are usually experimentally confounded, making it difficult to link neural observations to specific processes. Here we separated the effects of selective attention from the effects of decision-making on brain activity obtained from human participants (both sexes), using a two-stage task where the attended stimulus and decision were orthogonal and separated in time. Multivariate pattern analyses of multimodal neuroimaging data revealed the dynamics of perceptual and decision-related information coding through time with magnetoencephalography (MEG), through space with functional magnetic resonance imaging (fMRI), and their combination (MEG-fMRI fusion). Our MEG results showed an effect of attention before decision-making could begin, and fMRI results showed an attention effect in early visual and frontoparietal regions. Model-based MEG-fMRI fusion suggested that attention boosted stimulus information in the frontoparietal and early visual regions before decision-making was possible. Together, our results suggest that attention affects neural stimulus representations in the frontoparietal regions independent of decision-making.

Neural representational geometries reflect behavioral differences in monkeys and recurrent neural networks

Animals likely use a variety of strategies to solve laboratory tasks. Traditionally, combined analysis of behavioral and neural recording data across subjects employing different strategies may obscure important signals and give confusing results. Hence, it is essential to develop techniques that can infer strategy at the single-subject level. We analyzed an experiment in which two male monkeys performed a visually cued rule-based task. The analysis of their performance shows no indication that they used a different strategy. However, when we examined the geometry of stimulus representations in the state space of the neural activities recorded in dorsolateral prefrontal cortex, we found striking differences between the two monkeys. Our purely neural results induced us to reanalyze the behavior. The new analysis showed that the differences in representational geometry are associated with differences in the reaction times, revealing behavioral differences we were unaware of. All these analyses suggest that the monkeys are using different strategies. Finally, using recurrent neural network models trained to perform the same task, we show that these strategies correlate with the amount of training, suggesting a possible explanation for the observed neural and behavioral differences.

Rhythmic variance influences the speed but not the accuracy of complex averaging decisions.

When a rhythm makes an event predictable, that event is perceived faster, and typically more accurately. However, the experiments showing this used simple tasks, and most manipulated temporal expectancy by using periodic or aperiodic precursors unrelated to stimulus and task. Three experiments tested the generality of these observations in a complex task in which rhythm was intrinsic to, rather than a precursor of, the information needed to respond: listeners averaged the laterality of a stream of noise bursts. We varied presentation rate, degree of periodicity, and average lateralisation. Decisions following a probe tone were fastest after periodic stimuli, and slowest after the most aperiodic stimuli. Without a probe tone, listeners responded sooner during periodic sequences, thus hearing less information. Periodicity did not benefit accuracy overall. This gain in speed but not accuracy for less information is not reported for simpler tasks. Neural entrainment supplemented by cognitive factors provide a tentative explanation. When the task is inherently complex and demands high attention over long durations, both expected-periodic and unexpected-aperiodic stimuli can increase response amplitude, enhancing stimulus representation, but periodicity increases confidence to respond early. Drift diffusion modelling supports this proposal: aperiodicity modulated the decision threshold, but not the drift rate or non-decision time. Together, these new data and the literature point towards task-dependent effects of temporal expectation on decision-making, showing interactions between rhythmic variance, task complexity, and sources of expectation about stimuli. We suggest the implications are worth exploring to extend understanding of rhythmicity on decision-making to everyday situations.