Representations In Prefrontal Cortex Research Articles

The neural architecture of theory-based reinforcement learning

Humans learn internal models of the world that support planning and generalization in complex environments. Yet it remains unclear how such internal models are represented and learned in the brain. We approach this question using theory-based reinforcement learning, a strong form of model-based reinforcement learning in which the model is a kind of intuitive theory. We analyzed fMRI data from human participants learning to play Atari-style games. We found evidence of theory representations in prefrontal cortex and of theory updating in prefrontal cortex, occipital cortex, and fusiform gyrus. Theory updates coincided with transient strengthening of theory representations. Effective connectivity during theory updating suggests that information flows from prefrontal theory-coding regions to posterior theory-updating regions. Together, our results are consistent with a neural architecture in which top-down theory representations originating in prefrontal regions shape sensory predictions in visual areas, where factored theory prediction errors are computed and trigger bottom-up updates of the theory.

Long-term learning transforms prefrontal cortex representations during working memory

The role of the lateral prefrontal cortex (lPFC) in working memory (WM) is debated. Non-human primate (NHP) electrophysiology shows that the lPFC stores WM representations, but human neuroimaging suggests that the lPFC controls WM content in sensory cortices. These accounts are confounded by differences in task training and stimulus exposure. We tested whether long-term training alters lPFC function by densely sampling WM activity using functional MRI. Over 3months, participants trained on both a WM and serial reaction time (SRT) task, wherein fractal stimuli were embedded within sequences. WM performance improved for trained (but not novel) fractals and, neurally, delay activity increased in distributed lPFC voxels across learning. Item-level WM representations became detectable within lPFC patterns, and lPFC activity reflected sequence relationships from the SRT task. These findings demonstrate that human lPFC develops stimulus-selective responses with learning, and WM representations are shaped by long-term experience, which could reconcile competing accounts of WM functioning.

Dynamic and stable population coding of attentional instructions coexist in the prefrontal cortex

A large body of recent work suggests that neural representations in prefrontal cortex (PFC) are changing over time to adapt to task demands. However, it remains unclear whether and how such dynamic coding schemes depend on the encoded variable and are influenced by anatomical constraints. Using a cued attention task and multivariate classification methods, we show that neuronal ensembles in PFC encode and retain in working memory spatial and color attentional instructions in an anatomically specific manner. Spatial instructions could be decoded both from the frontal eye field (FEF) and the ventrolateral PFC (vlPFC) population, albeit more robustly from FEF, whereas color instructions were decoded more robustly from vlPFC. Decoding spatial and color information from vlPFC activity in the high-dimensional state space indicated stronger dynamics for color, across the cue presentation and memory periods. The change in the color code was largely due to rapid changes in the network state during the transition to the delay period. However, we found that dynamic vlPFC activity contained time-invariant color information within a low-dimensional subspace of neural activity that allowed for stable decoding of color across time. Furthermore, spatial attention influenced decoding of stimuli features profoundly in vlPFC, but less so in visual area V4. Overall, our results suggest that dynamic population coding of attentional instructions within PFC is shaped by anatomical constraints and can coexist with stable subspace coding that allows time-invariant decoding of information about the future target.

Partially overlapping spatial environments trigger reinstatement in hippocampus and schema representations in prefrontal cortex

When we remember a city that we have visited, we retrieve places related to finding our goal but also non-target locations within this environment. Yet, understanding how the human brain implements the neural computations underlying holistic retrieval remains unsolved, particularly for shared aspects of environments. Here, human participants learned and retrieved details from three partially overlapping environments while undergoing high-resolution functional magnetic resonance imaging (fMRI). Our findings show reinstatement of stores even when they are not related to a specific trial probe, providing evidence for holistic environmental retrieval. For stores shared between cities, we find evidence for pattern separation (representational orthogonalization) in hippocampal subfield CA2/3/DG and repulsion in CA1 (differentiation beyond orthogonalization). Additionally, our findings demonstrate that medial prefrontal cortex (mPFC) stores representations of the common spatial structure, termed schema, across environments. Together, our findings suggest how unique and common elements of multiple spatial environments are accessed computationally and neurally.

Executive Function Assigns Value to Novel Goal-Congruent Outcomes.

People often learn from the outcomes of their actions, even when these outcomes do not involve material rewards or punishments. How does our brain provide this flexibility? We combined behavior, computational modeling, and functional neuroimaging to probe whether learning from abstract novel outcomes harnesses the same circuitry that supports learning from familiar secondary reinforcers. Behavior and neuroimaging revealed that novel images can act as a substitute for rewards during instrumental learning, producing reliable reward-like signals in dopaminergic circuits. Moreover, we found evidence that prefrontal correlates of executive control may play a role in shaping flexible responses in reward circuits. These results suggest that learning from novel outcomes is supported by an interplay between high-level representations in prefrontal cortex and low-level responses in subcortical reward circuits. This interaction may allow for human reinforcement learning over arbitrarily abstract reward functions.

Shared mechanisms underlie the control of working memory and attention.

Cognitive control guides behaviour by controlling what, when, and how information is represented in the brain1. For example, attention controls sensory processing; top-down signals from prefrontal and parietal cortex strengthen the representation of task-relevant stimuli2-4. A similar 'selection' mechanism is thought to control the representations held 'in mind'-in working memory5-10. Here we show that shared neural mechanisms underlie the selection of items from working memory and attention to sensory stimuli. We trained rhesus monkeys to switch between two tasks, either selecting one item from a set of items held in working memory or attending to one stimulus from a set of visual stimuli. Neural recordings showed that similar representations in prefrontal cortex encoded the control of both selection and attention, suggesting that prefrontal cortex acts as a domain-general controller. By contrast, both attention and selection were represented independently in parietal and visual cortex. Both selection and attention facilitated behaviour by enhancing and transforming the representation of the selected memory or attended stimulus. Specifically, during the selection task, memory items were initially represented in independent subspaces of neural activity in prefrontal cortex. Selecting an item caused its representation to transform from its own subspace to a new subspace used to guide behaviour. A similar transformation occurred for attention. Our results suggest that prefrontal cortex controls cognition by dynamically transforming representations to control what and when cognitive computations are engaged.

Hemispheres in harmony

In this issue of Neuron, Brincat etal. (2021) track in real time the working memory representations in prefrontal cortex of both hemispheres, as a saccade moves the remembered location from one hemifield to the other. They reveal a soft interhemispheric handover subserved by local rhythms and their interhemispheric synchronization.

Unconscious reinforcement learning of hidden brain states supported by confidence

Can humans be trained to make strategic use of latent representations in their own brains? We investigate how human subjects can derive reward-maximizing choices from intrinsic high-dimensional information represented stochastically in neural activity. Reward contingencies are defined in real-time by fMRI multivoxel patterns; optimal action policies thereby depend on multidimensional brain activity taking place below the threshold of consciousness, by design. We find that subjects can solve the task within two hundred trials and errors, as their reinforcement learning processes interact with metacognitive functions (quantified as the meaningfulness of their decision confidence). Computational modelling and multivariate analyses identify a frontostriatal neural mechanism by which the brain may untangle the ‘curse of dimensionality’: synchronization of confidence representations in prefrontal cortex with reward prediction errors in basal ganglia support exploration of latent task representations. These results may provide an alternative starting point for future investigations into unconscious learning and functions of metacognition.

Retrieval practice facilitates memory updating by enhancing and differentiating medial prefrontal cortex representations.

Updating old memories with new, more current information is critical for human survival, yet the neural mechanisms for memory updating in general and the effect of retrieval practice in particular are poorly understood. Using a three-day A-B/A-C memory updating paradigm, we found that compared to restudy, retrieval practice could strengthen new A-C memories and reduce old A-B memory intrusion, but did not suppress A-B memories. Neural activation pattern analysis revealed that compared to restudy, retrieval practice led to stronger target representation in the medial prefrontal cortex (MPFC) during the final test. Critically, it was only under the retrieval practice condition that the MPFC showed strong and comparable competitor evidence for both correct and incorrect trials during final test, and that the MPFC target representation during updating was predictive of subsequent memory. These results suggest that retrieval practice is able to facilitate memory updating by strongly engaging MPFC mechanisms in memory integration, differentiation and consolidation.

Associations between sounds and actions in primate prefrontal cortex

Behavioral flexibility allows animals to cope with changing situations, for example, to execute different actions to the same stimulus to achieve specific goals in different situations. The selection of the appropriate action in a given situation hinges on the previously learned associations between stimuli, actions, and outcomes. We showed in our recent study that early auditory cortex of nonhuman primates contributes to the selection of the actions to sounds by representing the associations between sounds and actions. That is, neurons in auditory cortex respond differently to a given sound when it signals different actions that are required to obtain a reward. Here, using the same monkey and the same tasks, we investigated whether the ventrolateral part of prefrontal cortex also represents such audiomotor associations as well as whether and how these representations differ from those in auditory cortex. Mirroring auditory cortex, neuronal responses to a given sound in prefrontal cortex changed with audiomotor associations, and the neuronal responses were largest when the sound signaled a no-go response. These findings suggest that prefrontal cortex also represents audiomotor associations and thus contributes to the selection of the actions to sounds during goal-directed behavior. The neuronal activity related to audiomotor associations started later in prefrontal cortex than in auditory cortex, suggesting that the representations in prefrontal cortex may originate in auditory cortex or in earlier stages of the auditory system.

Thalamic regulation of switching between cortical representations enables cognitive flexibility.

Interactions between the prefrontal cortex (PFC) and mediodorsal thalamus (MD) are critical for cognitive flexibility, yet the underlying computations are unknown. To investigate fronto-thalamic substrates of cognitive flexibility, we developed a behavioral task, where mice switched between different sets of learned cues that guided attention towards either visual or auditory targets. We found that PFC responses reflected both the individual cues and their meaning as task rules, indicating a hierarchical cue-to-rule transformation. Conversely, MD responses reflected the statistical regularity of cue presentation, and were required for switching between such experimentally-specified cueing contexts. A subset of these thalamic responses sustained context-relevant PFC representations, while another suppressed the context-irrelevant ones. Through modeling and experimental validation, we find that thalamic-mediated suppression may not only reduce PFC representational interference but could also preserve unused cortical traces for future use. Overall, our study provides a computational foundation for thalamic engagement in cognitive flexibility.

Triple dissociation of attention and decision computations across prefrontal cortex.

Naturalistic decision-making typically involves sequential deployment of attention to choice alternatives to gather information before a decision is made. Attention filters how information enters decision circuits, implying attentional control may shape how decision computations unfold. We recorded neuronal activity from three subregions of prefrontal cortex (PFC) while monkeys performed an attention-guided decision-making task. From the first saccade to decision-relevant information, a triple dissociation of decision- and attention-related computations emerged in parallel across PFC subregions. During subsequent saccades, orbitofrontal cortex activity reflected value comparison between currently and previously attended information. By contrast, anterior cingulate cortex carried several signals reflecting belief updating in light of newly attended information, integration of evidence to a decision bound, and an emerging plan for what action to choose. Our findings show how anatomically dissociable PFC representations evolve during attention-guided information search, supporting computations critical for value-guided choice.

Modality-Independent Coding of Scene Categories in Prefrontal Cortex.

Natural environments convey information through multiple sensory modalities, all of which contribute to people's percepts. Although it has been shown that visual or auditory content of scene categories can be decoded from brain activity, it remains unclear how humans represent scene information beyond a specific sensory modality domain. To address this question, we investigated how categories of scene images and sounds are represented in several brain regions. A group of healthy human subjects (both sexes) participated in the present study, where their brain activity was measured with fMRI while viewing images or listening to sounds of different real-world environments. We found that both visual and auditory scene categories can be decoded not only from modality-specific areas, but also from several brain regions in the temporal, parietal, and prefrontal cortex (PFC). Intriguingly, only in the PFC, but not in any other regions, categories of scene images and sounds appear to be represented in similar activation patterns, suggesting that scene representations in PFC are modality-independent. Furthermore, the error patterns of neural decoders indicate that category-specific neural activity patterns in the middle and superior frontal gyri are tightly linked to categorization behavior. Our findings demonstrate that complex scene information is represented at an abstract level in the PFC, regardless of the sensory modality of the stimulus.SIGNIFICANCE STATEMENT Our experience in daily life includes multiple sensory inputs, such as images, sounds, or scents from the surroundings, which all contribute to our understanding of the environment. Here, for the first time, we investigated where and how in the brain information about the natural environment from multiple senses is merged to form modality-independent representations of scene categories. We show direct decoding of scene categories across sensory modalities from patterns of neural activity in the prefrontal cortex (PFC). We also conclusively tie these neural representations to human categorization behavior by comparing patterns of errors between a neural decoder and behavior. Our findings suggest that PFC is a central hub for integrating sensory information and computing modality-independent representations of scene categories.

Feedback from human posterior parietal cortex enables visuospatial category representations as early as primary visual cortex.

IntroductionCategorization is a fundamental cognitive process, whereby the brain assigns meaning to sensory stimuli. Previous studies have found category representations in prefrontal cortex and posterior parietal cortex (PPC). However, these higher‐order areas lack the fine‐scale spatial representations of early sensory areas, and it remains unclear what mechanisms enable flexible categorization based on fine‐scale features.MethodsIn this study, we decoded functional MRI signals and measured causal influences, across visual, parietal, and prefrontal cortex from participants performing categorization based on coarse‐ or fine‐scale spatial information in thirteen healthy adults.ResultsWe show that category information based on coarse discriminations was represented in the PPC, in the intraparietal sulcus region, IPS1/2, at an early stage of categorization trials, whereas representations of category information based on fine‐scale discriminations formed later during interactions between IPS1/2 and primary visual cortex (V1). Specifically, when fine‐scale discriminations were necessary, we decoded significant category information from V1 at an intermediate stage of trials and again from IPS1/2 at a late stage. IPS1/2 feedback was critical, because categorization performance improved as causal influence from IPS1/2 to V1 increased. Further, these mechanisms were plastic, as the selectivity of IPS1/2 and V1 responses shifted markedly with retraining to categorize the same stimuli into two new groups.ConclusionsOur findings suggest that reentrant processing between the PPC and visual cortex enables flexible abstraction of category information.

Fixation target representation in prefrontal cortex during the antisaccade task.

Neurons that discharge strongly during the time period of fixation of a visual target and cease to discharge before saccade initiation have been described in the brain stem, superior colliculus, and cortical areas. In subcortical structures, fixation neurons play a reciprocal role with saccadic neurons during the generation of eye movements. Their role in the dorsolateral prefrontal cortex is less obvious, and it is not known if they are activated by fixation, inhibit saccade generation, or play a role in more complex functions such as the inhibition of inappropriate responses. We examined the properties of prefrontal fixation neurons in the context of an antisaccade task, which requires an eye movement directed away from a prepotent visual stimulus. We tested monkeys with variants of the task, allowing us to dissociate activity synchronized on the fixation offset, presentation of the visual stimulus, and saccadic onset. Fixation neuron activity latency was most strongly tied to the offset of the fixation point across task variants. It was not well predicted by the appearance of the visual stimulus, which is essential for planning of the correct eye movement and inhibiting inappropriate ones. Activity of fixation neurons was generally negatively correlated with that of saccade neurons; however, critical differences in timing make it unlikely that they provide precisely timed signals for the generation of eye movements. These results demonstrate the role of fixation neurons in the prefrontal cortex during tasks requiring timing of appropriate eye movement and inhibition of inappropriate actions.NEW & NOTEWORTHY Properties of neurons that discharge during eye fixation and go silent before saccade initiation have been described in subcortical structures involved in eye movement generation, but their role in the dorsolateral prefrontal cortex presents a puzzle. Our results demonstrate the role of fixation neurons in the prefrontal cortex during tasks requiring precise timing of appropriate eye movement and inhibition of inappropriate actions.

Idiosyncratic Patterns of Representational Similarity in Prefrontal Cortex Predict Attentional Performance.

Everyone's perception of the world is uniquely shaped by personal experiences and preferences. Using functional MRI, we show that individual differences in the categorization of face morphs between two identities could be decoded from the prefrontal cortex and the ventral temporal cortex. Moreover, the individually unique representations in prefrontal cortex predicted idiosyncratic variability in attentional performance when looking for each identity in the "crowd" of another morphed face in a separate search task. Our results reveal that the representation of task-related information in prefrontal cortex is individually unique and preserved across categorization and search performance. This demonstrates the possibility of predicting individual behaviors across tasks with patterns of brain activity.

Behavioral response inhibition and maturation of goal representation in prefrontal cortex after puberty

Executive functions including behavioral response inhibition mature after puberty, in tandem with structural changes in the prefrontal cortex. Little is known about how activity of prefrontal neurons relates to this profound cognitive development. To examine this, we tracked neuronal responses of the prefrontal cortex in monkeys as they transitioned from puberty into adulthood and compared activity at different developmental stages. Performance of the antisaccade task greatly improved in this period. Among neural mechanisms that could facilitate it, reduction of stimulus-driven activity, increased saccadic activity, or enhanced representation of the opposing goal location, only the latter was evident in adulthood. Greatly accentuated in adults, this neural correlate of vector inversion may be a prerequisite to the formation of a motor plan to look away from the stimulus. Our results suggest that the prefrontal mechanisms that underlie mature performance on the antisaccade task are more strongly associated with forming an alternative plan of action than with suppressing the neural impact of the prepotent stimulus.

Neuronal activity in primate prefrontal cortex related to goal-directed behavior during auditory working memory tasks

Prefrontal cortex (PFC) has been documented to play critical roles in goal-directed behaviors, like representing goal-relevant events and working memory (WM). However, neurophysiological evidence for such roles of PFC has been obtained mainly with visual tasks but rarely with auditory tasks. In the present study, we tested roles of PFC in auditory goal-directed behaviors by recording local field potentials in the auditory region of left ventrolateral PFC while a monkey performed auditory WM tasks. The tasks consisted of multiple events and required the monkey to change its mental states to achieve the reward. The events were auditory and visual stimuli, as well as specific actions. Mental states were engaging in the tasks and holding task-relevant information in auditory WM. We found that, although based on recordings from one hemisphere in one monkey only, PFC represented multiple events that were important for achieving reward, including auditory and visual stimuli like turning on and off an LED, as well as bar touch. The responses to auditory events depended on the tasks and on the context of the tasks. This provides support for the idea that neuronal representations in PFC are flexible and can be related to the behavioral meaning of stimuli. We also found that engaging in the tasks and holding information in auditory WM were associated with persistent changes of slow potentials, both of which are essential for auditory goal-directed behaviors. Our study, on a single hemisphere in a single monkey, reveals roles of PFC in auditory goal-directed behaviors similar to those in visual goal-directed behaviors, suggesting that functions of PFC in goal-directed behaviors are probably common across the auditory and visual modality.This article is part of a Special Issue entitled SI: Auditory working memory.

The Representational Basis of Working Memory.

Working memory refers to a system involved in the online maintenance and manipulation of information in the absence of external input. Due to the importance of working memory in higher-level cognition, a wealth of neuroscience studies has investigated its neural basis. These studies have often led to conflicting viewpoints regarding the importance of the prefrontal cortex (PFC) and posterior sensory cortices. Here, we review evidence for each position. We suggest that the relative contributions of the PFC and sensory cortices to working memory can be understood with respect to processing demands. We argue that procedures that minimize processing demands lead to increased importance of sensory representations, while procedures that permit transformational processing lead to representational abstraction that relies on the PFC. We suggest that abstract PFC representations support top-down control over posterior representations while also providing bottom-up inputs into higher-level cognitive processing. Although a number of contemporary studies have studied working memory while using procedures that minimize the role of the PFC, we argue that consideration of the PFC is critical for our understanding of working memory and higher-level cognition more generally.

Age-related differences in the neural basis of the subjective vividness of memories: evidence from multivoxel pattern classification.

Although older adults often show reduced episodic memory accuracy, their ratings of the subjective vividness of their memories often equal or even exceed those of young adults. Such findings suggest that young and older adults may differentially access and/or weight different kinds of information in making vividness judgments. We examined this idea using multivoxel pattern classification of fMRI data to measure category representations while participants saw and remembered pictures of objects and scenes. Consistent with our hypothesis, there were age-related differences in how category representations related to the subjective sense of vividness. During remembering, older adults' vividness ratings were more related, relative to young adults', to category representations in prefrontal cortex. In contrast, young adults' vividness ratings were more related, relative to older adults, to category representations in parietal cortex. In addition, category representations were more correlated among posterior regions in young than in older adults, whereas correlations between PFC and posterior regions did not differ between the 2 groups. Together, these results are consistent with the idea that young and older adults differentially weight different types of information in assessing subjective vividness of their memories.