Location Of Reward Research Articles

Are We There Yet? Identification of Reward-Selective Cells in the Hippocampus

Navigation to a previously visited reward site requires a reliable and accurate spatial memory. In this issue ofNeuron, Gauthier and Tank (2018) use two-photon calcium imaging to uncover a discrete hippocampal subpopulation specialized for encoding reward location.

Low Birth Weight Impairs Acquisition of Spatial Memory Task in Pigs.

In commercial pig farming, an increasing number of low birth weight (LBW) piglets are born, due to selection for large litter sizes. While LBW piglets have a higher risk of pre-weaning mortality, a considerable number of these piglets survive to slaughter age. In humans, LBW is a risk factor for long-term cognitive impairments. In pigs, studies examining the post-weaning effects of LBW on cognition have reported contradictory results. Therefore, the current study aimed to assess the effects of LBW on cognitive development in pigs using an improved study design, by (1) testing a larger sample size than previous studies, (2) assessing acute and chronic stress responses to account for a potential altered stress response in LBW pigs, and (3) testing both female and male pigs to account for potential confounding effects of sex. Learning and memory of 20 LBW pigs and 20 normal birth weight (NBW) pigs, both groups consisting of 10 females and 10 males, were compared using a spatial holeboard task. In this task, pigs had to learn and remember the locations of hidden food rewards. After a pig had successfully acquired the task, it was presented with two successive reversal phases during which it was presented with a new configuration of reward locations. The holeboard allows for simultaneous assessment of working and reference memory, as well as measures of motivation, exploration, and behavioral flexibility. Mixed model ANOVAs revealed a transiently impaired reference memory performance of LBW pigs, implying they had more difficulty learning their reward configuration in the holeboard. Also, LBW piglets showed increased pre-weaning hair cortisol concentrations compared to their NBW siblings. No other effects of LBW were found. Sex had no direct or interaction effects on any measures of holeboard performance or stress. It is possible that the enriched housing conditions applied during our study had an ameliorating effect on our pigs' cognitive development. Overall, our results suggest LBW has a negative effect on post-weaning cognitive performance in pigs. This could have welfare consequences as cognitive skills are required for pigs to learn how to correctly respond to their environment.

Acetylcholine-modulated plasticity in reward-driven navigation: a computational study

Neuromodulation plays a fundamental role in the acquisition of new behaviours. In previous experimental work, we showed that acetylcholine biases hippocampal synaptic plasticity towards depression, and the subsequent application of dopamine can retroactively convert depression into potentiation. We also demonstrated that incorporating this sequentially neuromodulated Spike-Timing-Dependent Plasticity (STDP) rule in a network model of navigation yields effective learning of changing reward locations. Here, we employ computational modelling to further characterize the effects of cholinergic depression on behaviour. We find that acetylcholine, by allowing learning from negative outcomes, enhances exploration over the action space. We show that this results in a variety of effects, depending on the structure of the model, the environment and the task. Interestingly, sequentially neuromodulated STDP also yields flexible learning, surpassing the performance of other reward-modulated plasticity rules.

Corticosterone impairs flexible adjustment of spatial navigation in an associative place–reward learning task

Cognitive challenges are often accompanied by a discharge of stress hormones, which in turn modulate multiple brain areas. Among these, the medial temporal lobe and the prefrontal cortex are critically involved in high-order cognitive functions such as learning, memory, and decision-making. Previous studies assessing the effects of corticosterone on spatial memory found an increase or a decrease in performance depending on the timing of stress hormone discharge relative to the behavioral task. Most of these studies, however, made use of aversively motivated behaviors, whereas less is known about corticosteroid effects on flexible learning during reward-driven spatial navigation. To study how corticosterone modulates flexible spatial learning, we tested rats on a place-reward association task where hormone treatment was administered immediately after a session presenting a change in reward locations. The corticosterone-treated group showed delayed learning during the initial sessions and suboptimal memory consolidation throughout testing. Repeated training on the novel reward positions improved performance and eliminated differences from the control group. We conclude that a marked increase in plasma corticosterone levels immediately after training impairs the flexible formation of new place-reward associations.

A comparison of planners' site-selection intentions towards a sustainability reward program, location, and overall costs

ABSTRACTPrevious research indicates that environmental attitudes do not always translate into environmentally sustainable behavioral intention. The current research uses sustainability reward programs as incentives for people to have environmentally friendly behavioral intentions. Specifically, this study investigates how sustainability reward program affect meeting planners' site-selection intention in comparison with other factors such as location and overall costs. While the results of the study discover that the effect of sustainability reward programs was not significant for planners' site-selection intention, considerable differences were found on the effects of other factors between different types of planners. The findings provide important theoretical and practical implications.

Characterizing social behavior, activity, and associations between cognition and behavior upon social grouping of weaned dairy calves

Weaned dairy calves are commonly exposed to changing physical and social environments, and ability to adapt to novel management is likely to have performance and welfare implications. We characterized how behavioral responses of weaned heifer calves develop over time after introduction to a social group. Previously individually reared Holstein heifer calves (n = 15; 60 ± 5 d of age; mean ± standard deviation) were introduced in weekly cohorts (5 ± 3 new calves/wk) to an existing group on pasture (8 ± 2 calves/group). We measured activity and behavior on the day of initial introduction and after 1 wk, when calves were exposed to regrouping (addition of younger calves and removal of older calves from the pen). Upon introduction, calves had 2 to 3 times more visits to each region of the pasture; they also spent more time at the back of the pasture, closest to where they were introduced and furthest from the feeding area (25.13 vs. 9.63% of observation period, standard error = 5.04), compared with behavior after 1 wk. Calves also spent less time feeding (5.0 vs. 9.6% of observation period, standard error = 0.82) and self-grooming (0.52 vs. 1.31% of observation period; standard error = 0.20) and more time within 1 to 3 body lengths of another calf (16.3 vs. 11.9% of observation period, standard error = 2.3) when initially grouped. We also explored whether behavioral responses to initial postweaning grouping might be associated with individual differences in behavioral flexibility. To evaluate this, we assessed cognition of individually housed calves (n = 18) at 5 wk of age using a spatial discrimination task conducted in a T-maze to measure initial learning (ability to learn the location of a milk reward) and reversal learning (ability to relearn location of the milk reward when it was switched to opposite arm of the maze). Calves were categorized by reversal learning success (passed, n = 6, or failed, n = 8). Calves that passed the reversal learning stage of the cognitive task spent less time at the back of the pen (9.3 vs. 27.4% of observation period, standard error = 5.5) and tended to have lower latency to feed (121.8 vs. 306.2 min; standard error = 96.4) on the day of introduction compared with calves that failed reversal learning. Overall, we found that initial introduction to social grouping had a marked influence on behavior of weaned calves that decreased over time. Further, these results suggest that individual variability in cognitive ability may be predictive of behavioral responses and ability to adapt to a novel environment.

Modality-specific Subpopulations of Place Fields Coexist in the Hippocampus

Hippocampal place cells integrate signals from multiple sensory modalities. However, it is unclear how these different inputs are combined to generate place fields. We investigated how visual spatial cues and an animal's locomotion are integrated by CA3 place cells of Mongolian gerbils. While the animals moved on a virtual linear track, we adapted the gain between the visually projected environment and the treadmill movement. Place cells responded differently to this manipulation. In a subset, place fields were kept in accord with salient visual cues in the virtual environment or reward location, whereas in another subset, place fields were strongly influenced by locomotion. Theta phase precession was present and indistinguishable between the place field types. Theta compression remained intact under gain changes and extended over both types of place field. Hippocampal place cells thus retain strong influence from distinct input streams suggesting a role of the hippocampus CA3 as a multimodal associator on the theta time scale.

Rat anterior cingulate cortex recalls features of remote reward locations after disfavoured reinforcements.

The anterior cingulate cortex (ACC) encodes information supporting mnemonic and cognitive processes. We show here that a rat's position can be decoded with high spatiotemporal resolution from ACC activity. ACC neurons encoded the current state of the animal and task, except for brief excursions that sometimes occurred at target feeders. During excursions, the decoded position became more similar to a remote target feeder than the rat's physical position. Excursions recruited activation of neurons encoding choice and reward, and the likelihood of excursions at a feeder was inversely correlated with feeder preference. These data suggest that the excursion phenomenon was related to evaluating real or fictive choice outcomes, particularly after disfavoured reinforcements. We propose that the multiplexing of position with choice-related information forms a mental model isomorphic with the task space, which can be mentally navigated via excursions to recall multimodal information about the utility of remote locations.

Chimpanzees show some evidence of selectively acquiring information by using tools, making inferences, and evaluating possible outcomes.

Metacognition refers to thinking about one’s thinking or knowing what one knows. Research suggests that this ability is not unique to humans and may be shared with nonhuman animals. In particular, great apes have shown behaviors on a variety of tasks that are suggestive of metacognitive ability. Here we combine a metacognitive task, the information-seeking task, with tool use and variable forms of initial information provided to chimpanzees to explore how informational states impact behavioral responses in these apes. Three chimpanzees were presented with an apparatus that contained five locations where food could be hidden. If they pointed to the correct location, they received the reward, but otherwise they did not. We first replicated several existing findings using this method, and then tested novel hypotheses. The chimpanzees were given different types of information across the experiments. Sometimes, they were shown the location of the food reward. Other times, they were shown only one empty location, which was not useful information. The chimpanzees also could use a tool to search any of those locations before making a selection. Chimpanzees typically used the tool to search out the location of the reward when they could not already know where it was, but they did not use the tool when they already had been given that information. One chimpanzee made inferences about the location of hidden food, even when that food was never shown in that location. The final experiment involved hiding foods of differing preference values, and then presenting the chimpanzees with different initial knowledge states (i.e., where the best food was located, where the less-preferred food was located, or where no food was located). All chimpanzees used the tool when they needed to use it to find the best possible item on that trial, but responded by choosing a location immediately when they did not need the tool. This finding highlights that their behavior was not the result of a simple rule following such as pointing to where any food had been seen.

Autophosphorylation of αCaMKII affects social interactions in mice.

The α-Ca2+ /calmodulin-dependent protein kinase II (αCaMKII), a key regulator of the glutamatergic synapse, has been implicated in many psychiatric disorders characterized by social impairments. Here we tested whether autophosphorylation of αCaMKII at threonine 286, which prolongs the activity of the enzyme, affects social behaviors in mice. We observed that autophosphorylation-deficient (αCaMKII-T286A) mutant female mice showed abnormal social behaviors characterized by decreased social preference and interest in conspecifics of the same sex, as compared to their wild-type littermates. Moreover, we developed a mathematical approach to analyze social interactions in group-housed mice in the automated IntelliCages. Using this approach we observed that αCaMKII-T286A mutants show decreased levels of social interactions in a social group, as compared with WT mice. WT mice increased the frequency of close social interactions when learning about the location of the food reward. This phenomenon was absent in the mutants. Overall, our data indicates that autophosphorylation of αCaMKII affects social interactions.

Dentate network activity is necessary for spatial working memory by supporting CA3 sharp-wave ripple generation and prospective firing of CA3 neurons.

Complex spatial working memory (WM) tasks have been shown to require both hippocampal sharp wave ripple (SWR) activity and dentate gyrus (DG) neuronal activity. We therefore asked whether DG inputs to CA3 contribute to spatial WM by promoting SWR generation. Recordings from DG and CA3 while rats performed a dentate-dependent WM task on an 8-arm radial maze revealed that the activity of dentate neurons and the incidence rate of SWRs both increased during reward consumption. We then found reduced reward-related CA3 SWR generation without direct input from dentate granule neurons. Furthermore, CA3 cells with place fields in not-yet visited arms preferentially fired during SWRs at reward locations, and these prospective CA3 firing patterns were more pronounced for correct trials and were dentate dependent. These results indicate that coordination of CA3 neuronal activity patterns by DG is necessary for the generation of neuronal firing patterns that support goal-directed behavior and memory.

Older and wiser? Age differences in foraging and learning by an endangered passerine

Birds use cues when foraging to help relocate food resources, but natural environments provide many potential cues and choosing which to use may depend on previous experience. Young animals have less experience of their environment compared to adults, so may be slower to learn cues or may need to sample the environment more. Whether age influences cue use and learning has, however, received little experimental testing in wild animals. Here we investigate effects of age in a wild population of hihi (Notiomystis cincta), a threatened New Zealand passerine. We manipulated bird feeders using a novel colour cue to indicate a food reward; once hihi learned its location, we rotated the feeder to determine whether the birds followed the colour or returned to the previous location. Both age groups made fewer errors over trials and learned the location of the food reward, but juveniles continued to sample unrewarding locations more than adults. Following a second rotation, more adults preferred to forage from the hole indicated by the colour cue than juveniles, despite this no longer being rewarding. Overall, juveniles spent longer in the feeder arena to reach the same proportion of foraging time as adults. Combined, these results suggest that juveniles and adults may use an “explore and exploit” foraging strategy differently, and this affects how efficiently they forage. Further work is needed to understand how juveniles may compensate for their inexperience in learning and foraging strategies.

Overt and covert attention to location-based reward

Recent research on the impact of location-based reward on attentional orienting has indicated that reward factors play an influential role in spatial priority maps. The current study investigated whether and how reward associations based on spatial location translate from overt eye movements to covert attention. If reward associations can be tied to locations in space, and if overt and covert attention rely on similar overlapping neuronal populations, then both overt and covert attentional measures should display similar spatial-based reward learning. Our results suggest that location- and reward-based changes in one attentional domain do not lead to similar changes in the other. Specifically, although we found similar improvements at differentially rewarded locations during overt attentional learning, this translated to the least improvement at a highly rewarded location during covert attention. We interpret this as the result of an increased motivational link between the high reward location and the trained eye movement response acquired during learning, leading to a relative slowing during covert attention when the eyes remained fixated and the saccade response was suppressed. In a second experiment participants were not required to keep fixated during the covert attention task and we no longer observed relative slowing at the high reward location. Furthermore, the second experiment revealed no covert spatial priority of rewarded locations. We conclude that the transfer of location-based reward associations is intimately linked with the reward-modulated motor response employed during learning, and alternative attentional and task contexts may interfere with learned spatial priorities.

Visible spatial contiguity of social information and reward affects social learning in brown capuchins (Sapajus apella) and children (Homo sapiens).

Animal social learning is typically studied experimentally by the presentation of artificial foraging tasks. Although productive, results are often variable even for the same species. We present and test the hypothesis that one cause of variation is that spatial distance between rewards and the means of reward release causes conflicts for participants' attentional focus. We investigated whether spatial contiguity between a visible reward and the means of release would affect behavioral responses that evidence social learning, testing 21 brown capuchins (Sapajus apella), a much-studied species with variant evidence for social learning, and one hundred eighty 2- to 4-year-old human children (Homo sapiens), a benchmark species known for a strong social learning disposition. Participants were presented with a novel transparent apparatus where a reward was either proximal or distal to a demonstrated means of releasing it. A distal reward location decreased attention toward the location of the demonstration and impaired subsequent success in gaining rewards. Generally, the capuchins produced the alternative method to that demonstrated, whereas children copied the method demonstrated, although a distal reward location reduced copying in younger children. We conclude that some design features in common social learning tasks may significantly degrade the evidence for social learning. We have demonstrated this for 2 different primates but suggest that it is a significant factor to control for in social learning research across all taxa. (PsycINFO Database Record

Task Demands Predict a Dynamic Switch in the Content of Awake Hippocampal Replay

SummaryReactivation of hippocampal place cell sequences during behavioral immobility and rest has been linked with both memory consolidation and navigational planning. Yet it remains to be investigated whether these functions are temporally segregated, occurring during different behavioral states. During a self-paced spatial task, awake hippocampal replay occurring either immediately before movement toward a reward location or just after arrival at a reward location preferentially involved cells consistent with the current trajectory. In contrast, during periods of extended immobility, no such biases were evident. Notably, the occurrence of task-focused reactivations predicted the accuracy of subsequent spatial decisions. Additionally, during immobility, but not periods preceding or succeeding movement, grid cells in deep layers of the entorhinal cortex replayed coherently with the hippocampus. Thus, hippocampal reactivations dynamically and abruptly switch between operational modes in response to task demands, plausibly moving from a state favoring navigational planning to one geared toward memory consolidation.

A novel continuous inhibitory-control task: variation in individual performance by young pheasants (Phasianus colchicus)

Inhibitory control enables subjects to quickly react to unexpectedly changing external demands. We assessed the ability of young (8 weeks old) pheasants Phasianus colchicus to exert inhibitory control in a novel response-inhibition task that required subjects to adjust their movement in space in pursuit of a reward across changing target locations. The difference in latencies between trials in which the target location did and did not change, the distance travelled towards the initially indicated location after a change occurred, and the change-signal reaction time provided a consistent measure that could be indicative of a pheasant’s inhibitory control. Between individuals, there was a great variability in these measures; these differences were not correlated with motivation either to access the reward or participate in the test. However, individuals that were slower to reach rewards in trials when the target did not change exhibited evidence of stronger inhibitory control, as did males and small individuals. This novel test paradigm offers a potential assay of inhibitory control that utilises a natural feature of an animal’s behavioural repertoire, likely common to a wide range of species, specifically their ability to rapidly alter their trajectory when reward locations switch.

Sequential neuromodulation of Hebbian plasticity offers mechanism for effective reward-based navigation

Spike timing-dependent plasticity (STDP) is under neuromodulatory control, which is correlated with distinct behavioral states. Previously, we reported that dopamine, a reward signal, broadens the time window for synaptic potentiation and modulates the outcome of hippocampal STDP even when applied after the plasticity induction protocol (Brzosko et al., 2015). Here, we demonstrate that sequential neuromodulation of STDP by acetylcholine and dopamine offers an efficacious model of reward-based navigation. Specifically, our experimental data in mouse hippocampal slices show that acetylcholine biases STDP toward synaptic depression, whilst subsequent application of dopamine converts this depression into potentiation. Incorporating this bidirectional neuromodulation-enabled correlational synaptic learning rule into a computational model yields effective navigation toward changing reward locations, as in natural foraging behavior. Thus, temporally sequenced neuromodulation of STDP enables associations to be made between actions and outcomes and also provides a possible mechanism for aligning the time scales of cellular and behavioral learning.

Effect of flower perceptibility on spatial-reward associative learning by bumble bees

The associative learning of spatial and reward information is generally considered an adaptive behavior of foraging animals that collect food from renewable resources. However, learning may not always maximize foraging efficiency if choosing a high-reward location accurately based on learned information imposes a time cost on the forager (speed-accuracy trade-off). To examine the effect of speed-accuracy trade-offs on associative learning, we observed bumble bees, Bombus ignitus (Smith), foraging in mixed arrays of high- and low-rewarding artificial flowers under two conditions, i.e., arrays of small flowers where bees could not easily detect the next nearest flowers on leaving a flower and arrays of large flowers where bees could easily recognize the next nearest flower. When flowers were small, bees created foraging routes by selectively incorporating the locations of high-rewarding flowers with their experience. When flowers were large, bees flew between flowers more quickly than when flowers were small, creating foraging routes without accounting for the locations of high-rewarding flowers. Estimated foraging efficiency was higher when flowers were large than when they were small, at least until flower visitation number reached 3000, suggesting that rapid foraging might be a better choice than accurate foraging when individuals are able to locate flowers easily. These results suggest that associative learning of spatial and reward information might be a choice that foragers can apply according to the cost-benefit balance of learning. Previous studies focusing on spatial-reward associative learning in foraging animals assumed that foraging efficiency increased as the forager learned the locations of greater rewards. However, this study suggests that bumble bees learn the location of high-rewarding flowers depending on the cost–benefit balance of learning, irrespective of the opportunity or their aptitude for learning. Thus, learning might be a choice that foragers can apply according to the circumstances.

The integration of social influence and reward: Computational approaches and neural evidence.

Decades of research have established that decision-making is dramatically impacted by both the rewards an individual receives and the behavior of others. How do these distinct influences exert their influence on an individual's actions, and can the resulting behavior be effectively captured in a computational model? To address this question, we employed a novel spatial foraging game in which groups of three participants sought to find the most rewarding location in an unfamiliar two-dimensional space. As the game transitioned from one block to the next, the availability of information regarding other group members was varied systematically, revealing the relative impacts of feedback from the environment and information from other group members on individual decision-making. Both reward-based and socially-based sources of information exerted a significant influence on behavior, and a computational model incorporating these effects was able to recapitulate several key trends in the behavioral data. In addition, our findings suggest how these sources were processed and combined during decision-making. Analysis of reaction time, location of gaze, and functional magnetic resonance imaging (fMRI) data indicated that these distinct sources of information were integrated simultaneously for each decision, rather than exerting their influence in a separate, all-or-none fashion across separate subsets of trials. These findings add to our understanding of how the separate influences of reward from the environment and information derived from other social agents are combined to produce decisions.

Contrasting Social and Cognitive Accounts on Overimitation: The Role of Causal Transparency and Prior Experiences

Three experiments (N=100) examine the influence of causal information on overimitation. In Experiment 1, a transparent reward location reveals that the reward is unaffected by nonfunctional actions. When 5-year-olds observe an inefficient and subsequently an efficient strategy to retrieve a reward, they show overimitation in both phases-even though the reward is visible. In Experiment 2, children observe first the efficient then the inefficient strategy. The latter is always demonstrated communicatively, whereas the efficient strategy is presented communicatively (2a) or noncommunicatively (2b). Regardless of whether the efficient strategy is emphasized through communication or not, most children do not switch from the efficient to the inefficient strategy. Depending on the situation, children base their behavior on social motivations or causal information.