Location Of Reward Research Articles

A mechanical task for measuring sign- and goal-tracking in humans: A proof-of-concept study

Cue-based associative learning (i.e., Pavlovian conditioning) is a foundational component of behavior in almost all forms of animal life and may provide insight into individual differences in addiction liability. Cues can take on incentive-motivational properties (i.e., incentive salience) through Pavlovian learning. Extensive testing with non-human animals (primarily rats) has demonstrated significant variation among individuals in the behaviors this type of learning evokes. So-named “sign-trackers” and “goal-trackers” have been examined in many studies of non-human animals, but this work in humans is still a nascent area of research. In the present proof-of-concept study, we used a Pavlovian conditioned approach task to investigate human sign- and goal-tracking in emerging adults. Conditioned behaviors that developed over the course of the task were directed toward the reward-cue and toward the reward location. Participants’ eye-gaze and behavior during the task were submitted to a latent profile analysis, which revealed three groups defined as sign-trackers (n = 10), goal-trackers (n = 4), and intermediate responders (n = 36). Impulsivity was a significant predictor of the sign-tracking group relative to the goal-tracking group. The present study provides preliminary evidence that a simple procedure can produce learned Pavlovian conditioned approach behavior in humans. Though further investigation is required, findings provide a promising step toward the long-term goal of translating important insights gleaned from basic research into treatment strategies that can be applied to clinical populations.

Spatial representation by ramping activity of neurons in the retrohippocampal cortex

Neurons in the retrohippocampal cortices play crucial roles in spatial memory. Many retrohippocampal neurons have firing fields that are selectively active at specific locations, with memory for rewarded locations associated with reorganization of these firing fields. Whether this is the sole strategy for representing spatial memories is unclear. Here, we demonstrate that during a spatial memory task retrohippocampal neurons encode location through ramping activity that extends across segments of a linear track approaching and following a reward, with the rewarded location represented by offsets or switches in the slope of the ramping activity. Ramping representations could be maintained independently of trial outcome and cues marking the reward location, indicating that they result from recall of the track structure. When recorded in an open arena, neurons that generated ramping activity during the spatial memory task were more numerous than grid or border cells, with a majority showing spatial firing that did not meet criteria for classification as grid or border representations. Encoding of rewarded locations through offsets and switches in the slope of ramping activity also emerged in recurrent neural network models trained to solve a similar spatial memory task. Impaired performance of model networks following disruption of outputs from ramping neurons is consistent with this coding strategy supporting navigation to recalled locations of behavioral significance. Our results suggest that encoding of learned spaces by retrohippocampal networks employs both discrete firing fields and continuous ramping representations. We hypothesize that retrohippocampal ramping activity mediates readout of learned models for goal-directed navigation.

Hippocampal astrocytes encode reward location.

Astrocytic calcium dynamics has been implicated in the encoding of sensory information1-5, and modulation of calcium in astrocytes has been shown to affect behaviour6-10. However, longitudinal investigation of the real-time calcium activity of astrocytes in the hippocampus of awake mice is lacking. Here we used two-photon microscopy to chronically image CA1 astrocytes as mice ran in familiar or new virtual environments to obtain water rewards. We found that astrocytes exhibit persistent ramping activity towards the reward location in a familiar environment, but not in a new one. Shifting the reward location within a familiar environment also resulted in diminished ramping. After additional training, as the mice became familiar with the new context or new reward location, the ramping was re-established. Using linear decoders, we could predict the location of the mouse in a familiar environment from astrocyte activity alone. We could not do the same in a new environment, suggesting that the spatial modulation of astrocytic activity is experience dependent. Our results indicate that astrocytes can encode the expected reward location in spatial contexts, thereby extending their known computational abilities and their role in cognitive functions.

The cognitive costs and advantages of children's exposure to parental relationship instability: Testing an evolutionary-developmental hypothesis.

Guided by evolutionary-developmental models, this study tested the hypothesis that children's exposure to parental relationship instability, defined by initiation and dissolution of caregiver intimate relationships, has both costs in cognitive impairments and benefits in enhanced learning skills. Participants included 243 mothers and their preschool children (M age 4.60 years; 56% girls) from diverse racial and ethnic backgrounds (e.g., 46% Black; 19% Latinx). Consistent with hypotheses, higher levels of parental relationship instability during preschool predicted children's poorer performance on explicit, higher-order cognitive functioning tasks (e.g., IQ, working memory) and better performance in detecting reward probabilities in an implicit learning task 2 years later. Results of the piecewise latent growth curve analysis of the implicit learning task revealed that children experiencing greater family instability were able to more rapidly identify the locations of the hidden rewards in the early, rather than later, stages of the games. Additional findings supported the role of children's antagonistic representations of family relationships as an intermediary mechanism. More specifically, parental relationship instability significantly predicted higher levels of children's antagonistic representations of their families 2 years later after controlling for their earlier antagonistic representations and demographic covariates. Children's antagonistic representations, in turn, were concurrently linked with poorer explicit cognitive functioning and better implicit learning abilities when they were in first grade. The findings inform an understanding of cognitive tradeoffs of experiencing parental relationship instability and may have important implications for modifying educational and clinical programs to capitalize on the cognitive strengths of children who experience environmental unpredictability. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Social Learning of a Spatial Task by Observation Alone.

Interactions between conspecifics are central to the acquisition of useful memories in the real world. Observational learning, i.e., learning a task by observing the success or failure of others, has been reported in many species, including rodents. However, previous work in rats with NMDA-receptor blockade has shown that even extensive observation of an unexplored space through a clear barrier is not sufficient to generate a stable hippocampal representation of that space. This raises the question of whether rats can learn a spatial task in a purely observed space from watching a conspecific, and if so, does this somehow stabilize their hippocampal representation? To address these questions, we designed an observational spatial task in a two-part environment that is nearly identical to that of the aforementioned electrophysiological study, in which an observer rat watches a demonstrator animal to learn the location of a hidden reward. Our results demonstrate that rats do not need to physically explore an environment to learn a reward location, provided a conspecific demonstrates where it is. We also show that the behavioral memory is not affected by NMDA receptor blockade, suggesting that the spatial representation underlying the behavior has been consolidated by observation alone.

Eye movements reveal spatiotemporal dynamics of visually-informed planning in navigation.

Goal-oriented navigation is widely understood to depend upon internal maps. Although this may be the case in many settings, humans tend to rely on vision in complex, unfamiliar environments. To study the nature of gaze during visually-guided navigation, we tasked humans to navigate to transiently visible goals in virtual mazes of varying levels of difficulty, observing that they took near-optimal trajectories in all arenas. By analyzing participants' eye movements, we gained insights into how they performed visually-informed planning. The spatial distribution of gaze revealed that environmental complexity mediated a striking trade-off in the extent to which attention was directed towards two complimentary aspects of the world model: the reward location and task-relevant transitions. The temporal evolution of gaze revealed rapid, sequential prospection of the future path, evocative of neural replay. These findings suggest that the spatiotemporal characteristics of gaze during navigation are significantly shaped by the unique cognitive computations underlying real-world, sequential decision making.

Aberrant Phase Precession of Lateral Septal Cells in a Maternal Immune Activation Model of Schizophrenia Risk May Disrupt the Integration of Location with Reward.

Spatial memory and reward processing are known to be disrupted in schizophrenia. Since the lateral septum (LS) may play an important role in the integration of location and reward, we examined the effect of maternal immune activation (MIA), a known schizophrenia risk factor, on spatial representation in the rat LS. In support of a previous study, we found that spatial location is represented as a phase code in the rostral LS of adult male rats, so that LS cell spiking shifts systematically against the phase of the hippocampal, theta-frequency, local field potential as an animal moves along a track toward a reward (phase precession). Whereas shallow precession slopes were observed in control group cells, they were steeper in the MIA animals, such that firing frequently precessed across several theta cycles as the animal moved along the length of the apparatus, with subsequent ambiguity in the phase representation of location. Furthermore, an analysis of the phase trajectories of the control group cells revealed that the population tended to converge toward a common firing phase as the animal approached the reward location. This suggested that phase coding in these cells might signal both reward location and the distance to reward. By comparison, the degree of phase convergence in the MIA-group cells was weak, and the region of peak convergence was distal to the reward location. These findings suggest that a schizophrenia risk factor disrupts the phase-based encoding of location-reward relationships in the LS, potentially smearing reward representations across space.SIGNIFICANCE STATEMENT It is unclear how spatial or contextual information generated by hippocampal cells is converted to a code that can be used to signal reward location in regions, such as the VTA. Here we provide evidence that the firing phase of cells in the lateral septum, a region that links the two areas, may code reward location in the firing phase of cells. This phase coding is disrupted in a maternal immune activation model of schizophrenia risk such that representations of reward may be smeared across space in maternal immune activation animals. This could potentially underlie erroneous reward processing and misattribution of salience in schizophrenia.

Great apes and human children rationally monitor their decisions.

Several species can detect when they are uncertain about what decision to make-revealed by opting out of the choice, or by seeking more information before deciding. However, we do not know whether any nonhuman animals recognize when they need more information to make a decision because new evidence contradicts an already-formed belief. Here, we explore this ability in great apes and human children. First, we show that after great apes saw new evidence contradicting their belief about which of two rewards was greater, they stopped to recheck the evidence for their belief before deciding. This indicates the ability to keep track of the reasons for their decisions, or 'rational monitoring' of the decision-making process. Children did the same at 5 years of age, but not at 3 years. In a second study, participants formed a belief about a reward's location, but then a social partner contradicted them, by picking the opposite location. This time even 3-year-old children rechecked the evidence, while apes ignored the disagreement. While apes were sensitive only to the conflict in physical evidence, the youngest children were more sensitive to peer disagreement than conflicting physical evidence.

Encoding reward locations.

Encoding reward locations.

Distinct Age-Specific Effects on Olfactory Associative Learning in C57BL/6 Substrains.

C57BL/6 is the most widely used mouse strain in the laboratories. Two substrains of C57BL/6, C57BL/6J (B6J), and C57BL/6N (B6N) are well-known backgrounds for genetic modification and have been shown difference in quite a few tests, including open field test, rotarod test, and Morris water maze. However, difference between these two substrains in olfaction-dependent behaviors remains unknown. Here, we used olfactory two-alternative choice task, which is modified to have two training stages, to evaluate animals’ ability in instrumental learning and olfactory association. In the first (rule learning) stage, the mice were trained to use the operant chamber to collect water rewards. An odor cue was provided in the procedure, with no indication about reward locations. In the following (discrimination learning) stage, two odor cues were provided, with each indicating a specific water port. The animals were rewarded upon correct port choices following cue deliveries. We found that during young adulthood (7–10 weeks old), proportionally more B6J than B6N mice were able to pass rule learning (58.3% vs. 29.2%) and ultimately acquire this task (54.2% vs. 25%), with the two substrains showing similar pass rates in discrimination learning (92.9% vs. 85.7%). Surprisingly, at a more mature age (17 weeks old), this substrain difference disappeared. Mature B6N mice had a significant improvement in pass percentages of rule learning and overall task, whereas similar improvement was not observed in the B6J counterparts. Instead, mature B6J mice had an improved speed in rule learning and overall task. We further examined behavioral patterns of 8-week-old B6J and B6N mice in the olfactory habituation or dishabituation test. We observed normal olfactory habituation from subjects of both substrains, with the B6J mice exhibiting stronger investigative responses to newly presented odorants. These results reveal for the first time that B6J and B6N mice are different in acquisition processes of a behavioral task that requires instrumental learning and olfactory association, and that maturation appears to employ different effects on these two substrains during these processes. Furthermore, young adult B6J and B6N mice might be similar in olfactory habituation but different in the olfactory aspects of novelty seeking.

A nonlinear hidden layer enables actor-critic agents to learn multiple paired association navigation.

Navigation to multiple cued reward locations has been increasingly used to study rodent learning. Though deep reinforcement learning agents have been shown to be able to learn the task, they are not biologically plausible. Biologically plausible classic actor-critic agents have been shown to learn to navigate to single reward locations, but which biologically plausible agents are able to learn multiple cue-reward location tasks has remained unclear. In this computational study, we show versions of classic agents that learn to navigate to a single reward location, and adapt to reward location displacement, but are not able to learn multiple paired association navigation. The limitation is overcome by an agent in which place cell and cue information are first processed by a feedforward nonlinear hidden layer with synapses to the actor and critic subject to temporal difference error-modulated plasticity. Faster learning is obtained when the feedforward layer is replaced by a recurrent reservoir network.

Reorganization of CA1 dendritic dynamics by hippocampal sharp-wave ripples during learning

The hippocampus plays a critical role in memory consolidation, mediated by coordinated network activity during sharp-wave ripple (SWR) events. Despite the link between SWRs and hippocampal plasticity, little is known about how network state affects information processing in dendrites, the primary sites of synaptic input integration and plasticity. Here, we monitored somatic and basal dendritic activity in CA1 pyramidal cells in behaving mice using longitudinal two-photon calcium imaging integrated with simultaneous local field potential recordings. We found immobility was associated with an increase in dendritic activity concentrated during SWRs. Coincident dendritic and somatic activity during SWRs predicted increased coupling during subsequent exploration of a novel environment. In contrast, somatic-dendritic coupling and SWR recruitment varied with cells' tuning distance to reward location during a goal-learning task. Our results connect SWRs with the stabilization of information processing within CA1 neurons and suggest that these mechanisms may be dynamically biased by behavioral demands.

Different encoding of reward location in dorsal and intermediate hippocampus

Hippocampal place cells fire at specific locations in the environment. They form a cognitive map that encodes spatial relations in the environment, including reward locations.1 As part of this encoding, dorsal CA1 (dCA1) place cells accumulate at reward.2-5 The encoding of learned reward location could vary between the dorsal and intermediate hippocampus, which differ in gene expression and cortical and subcortical connectivity.6 While the dorsal hippocampus is critical for spatial navigation, the involvement of intermediate CA1 (iCA1) in spatial navigation might depend on task complexity7 and learning phase.8-10 The intermediate-to-ventral hippocampus regulates reward-seeking,11-15 but little is known about the involvement in reward-directed navigation. Here, we compared the encoding of learned reward locations in dCA1 and iCA1 during spatial navigation. We used calcium imaging with a head-mounted microscope to track the activity of CA1 cells over multiple days during which mice learned different reward locations. In dCA1, the fraction of active place cells increased in anticipation of reward, but the pool of active cells changed with the reward location. In iCA1, the same cells anticipated multiple reward locations. Our results support a model in which the dCA1 cognitive map incorporates a changing population of cells that encodes reward proximity through increased population activity, while iCA1 provides a reward-predictive code through a dedicated subpopulation. Both of these location-invariant codes persisted over time, and together they provide a dual hippocampal reward location code, assisting goal-directed navigation.16,17.

A Neural Network Model of the Entorhinal Cortex and Hippocampus for Event-Order Memory Processing

To solve a navigation task based on experiences, we need a mechanism to associate places with objects and recall them along the course of action. In a reward-oriented task, if the route to a reward location is simulated in mind after experiencing it once, it might be possible that the reward is gained efficiently. One way to solve this is to incorporate a biologically plausible mechanism. In this study, we propose a neural network that stores a sequence of events associated with a reward. The proposed network recalls the reward location by tracing them in its mind in order. We simulated a virtual mouse that explores a figure-eight maze and recalls the route to the reward location. During the learning period, a sequence of events related to firing along a passage was temporarily stored in the heteroassociative network, and the sequence of events is consolidated in the synaptic weight matrix when a reward is fed. For retrieval, an impetus input internally generates the sequential activation of conjunctive cue–place cells toward the reward location. In the figure-eight maze task, the location of the reward was estimated by mind travel, irrespective of whether the reward is in the counterclockwise or distant clockwise route. The mechanism of efficiently reaching the goal by mind travel in the brain based on experiences is beneficial for mobile service robots that perform autonomous navigation.

Observational learning promotes hippocampal remote awake replay toward future reward locations

The neural circuit mechanisms underlying observational learning, learning through observing the behavior of others, are poorly understood. Hippocampal place cells are important for spatial learning, and awake replay of place cell patterns is involved in spatial decisions. Here we show that, in observer rats learning to run a maze by watching a demonstrator's spatial trajectories from a separate nearby observation box, place cell patterns during self-running in the maze are replayed remotely in the box. The contents of the remote awake replay preferentially target the maze's reward sites from both forward and reverse replay directions and reflect the observer's future correct trajectories in the maze. In contrast, under control conditions without a demonstrator, the remote replay is significantly reduced, and the preferences for reward sites and future trajectories disappear. Our results suggest that social observation directs the contents of remote awake replay to guide spatial decisions in observational learning.

Nicotine Enhances Goal-Tracking in Ethanol and Food Pavlovian Conditioned Approach Paradigms.

RationaleNicotine promotes alcohol intake through pharmacological and behavioral interactions. As an example of the latter, nicotine can facilitate approach toward food- and alcohol-associated stimuli (“sign-tracking”) in lever-Pavlovian conditioned approach (PavCA) paradigms. However, we recently reported that nicotine can also enhance approach toward locations of reward delivery (“goal-tracking”) triggered by ethanol-predictive stimuli when the location of ethanol delivery is non-static (i.e., a retractable sipper bottle).ObjectiveTo determine whether the non-static nature of the reward location could have biased the development of goal-tracking in our previous study (Loney et al., 2019); we assessed the effect of nicotine in a lever-PavCA paradigm wherein the location of ethanol delivery was static (i.e., a stationary liquid receptacle). Then, to determine whether nicotine’s enhancement of goal-tracking is unique to ethanol-predictive stimuli, we assessed the effect of systemic nicotine on approach triggered by food-predictive stimuli in a lever-PavCA paradigm.MethodsLong–Evans rats were used in two PavCA experiments wherein a lever predicted the receipt of ethanol (15% vol/vol; experiment 1) or food (experiment 2) into a stationary receptacle. Prior to testing, rats were administered nicotine (0.4 mg/kg subcutaneously) or saline systemically.ResultsIn both experiments, nicotine increased measures of goal-tracking, but not sign-tracking.ConclusionNicotine can facilitate approach to reward locations without facilitating approach to reward-predictive stimuli. As such, conceptualization of the mechanisms by which nicotine affects behavior must be expanded to explain an enhancement of goal-tracking by nicotine.

Dorsolateral Striatal Task-initiation Bursts Represent Past Experiences More than Future Action Plans.

The dorsolateral striatum (DLS) is involved in learning and executing procedural actions. Cell ensembles in the DLS, but not the dorsomedial striatum (DMS), exhibit a burst of firing at the start of a well-learned action sequence ("task-bracketing"). However, it is currently unclear what information is contained in these bursts. Some theories suggest that these bursts should represent the procedural action sequence itself (that they should be about future action chains), whereas others suggest that they should contain representations of the current state of the world, taking into account primarily past information. In addition, the DLS local field potential shows transient bursts of power in the 50 Hz range (γ50) around the time a learned action sequence is initiated. However, it is currently unknown how bursts of activity in DLS cell ensembles and bursts of γ50 power in the DLS local field potential are related to each other. We found that DLS bursts at lap initiation in rats represented recently experienced reward locations more than future procedural actions, indicating that task-initiation DLS bursts contain primarily retrospective, rather than prospective, information to guide procedural actions. Furthermore, representations of past reward locations increased during periods of increased γ50 power in the DLS. There was no evidence of task-initiation bursts, increased γ50 power, or retrospective reward location information in the neighboring dorsomedial striatum. These data support a role for the DLS in model-free theories of procedural decision-making over planned action-chain theories, suggesting that procedural actions derive from representations of the current and recent past.SIGNIFICANCE STATEMENT While it is well-established that the dorsolateral striatum (DLS) plays a critical role in procedural decision-making, open questions remain about the kinds of representations contained in DLS ensemble activity that guide procedural actions. We found that DLS, but not DMS, cell ensembles contained nonlocal representations of past reward locations that appear moments before task-initiation DLS bursts. These retrospective representations were temporally linked to a rise in γ50 power that also preceded the characteristic DLS burst at task-initiation. These results support models of procedural decision-making based on associations between available actions and the current state of the world over models based on planning over action-chains.

Effects of Birth Order on Performance and Affective State of Pigs

In pigs, higher birth order is associated with higher pre-weaning mortality. However, knowledge on the effect of birth order on welfare of surviving piglets is limited. The aim of this study was to explore the possible link between birth order and both newborn piglet performance and later affective state. Firstly, the following data were collected in 393 piglets from 27 litters: stillbirth, intactness of the umbilical cord and birth weight. Subsets of piglets were used to measure suckling latency (n = 67) and teat order (n = 21). Secondly, a subset of first-born (n = 9) and last-born (n = 7) piglets were trained to perform an active-choice judgement bias task (JBT). During discrimination training preceding the JBT, the pigs learned to associate two tone cues with the availability of either a large (4 M&M's® Milk Chocolate candies) or small (1 M&M's) reward, provided at two different locations. After training, ambiguous intermediate tones were introduced and the pig's choice of location was scored as either optimistic or pessimistic. Results showed that last-born piglets had a higher birth weight than middle-born piglets, while first-born piglets did not significantly differ from last- or middle-born piglets. They also latched to more caudal teats than first-born piglets. The last-born and first-born piglets showed a similar learning rate during discrimination training, and a similar latency to approach reward locations, and had a similar optimistic bias in the JBT.

Rest Is Required to Learn an Appetitively-Reinforced Operant Task in Drosophila.

Maladaptive operant conditioning contributes to development of neuropsychiatric disorders. Candidate genes have been identified that contribute to this maladaptive plasticity, but the neural basis of operant conditioning in genetic model organisms remains poorly understood. The fruit fly Drosophila melanogaster is a versatile genetic model organism that readily forms operant associations with punishment stimuli. However, operant conditioning with a food reward has not been demonstrated in flies, limiting the types of neural circuits that can be studied. Here we present the first sucrose-reinforced operant conditioning paradigm for flies. In the paradigm, flies walk along a Y-shaped track with reward locations at the terminus of each hallway. When flies turn in the reinforced direction at the center of the track, they receive a sucrose reward at the end of the hallway. Only flies that rest early in training learn the reward contingency normally. Flies rewarded independently of their behavior do not form a learned association but have the same amount of rest as trained flies, showing that rest is not driven by learning. Optogenetically-induced sleep does not promote learning, indicating that sleep itself is not sufficient for learning the operant task. We validated the sensitivity of this assay to detect the effect of genetic manipulations by testing the classic learning mutant dunce. Dunce flies are learning-impaired in the Y-Track task, indicating a likely role for cAMP in the operant coincidence detector. This novel training paradigm will provide valuable insight into the molecular mechanisms of disease and the link between sleep and learning.

The functional role of sequentially neuromodulated synaptic plasticity in behavioural learning.

To survive, animals have to quickly modify their behaviour when the reward changes. The internal representations responsible for this are updated through synaptic weight changes, mediated by certain neuromodulators conveying feedback from the environment. In previous experiments, we discovered a form of hippocampal Spike-Timing-Dependent-Plasticity (STDP) that is sequentially modulated by acetylcholine and dopamine. Acetylcholine facilitates synaptic depression, while dopamine retroactively converts the depression into potentiation. When these experimental findings were implemented as a learning rule in a computational model, our simulations showed that cholinergic-facilitated depression is important for reversal learning. In the present study, we tested the model’s prediction by optogenetically inactivating cholinergic neurons in mice during a hippocampus-dependent spatial learning task with changing rewards. We found that reversal learning, but not initial place learning, was impaired, verifying our computational prediction that acetylcholine-modulated plasticity promotes the unlearning of old reward locations. Further, differences in neuromodulator concentrations in the model captured mouse-by-mouse performance variability in the optogenetic experiments. Our line of work sheds light on how neuromodulators enable the learning of new contingencies.