Nonrewarded Trials Research Articles

Neural bases of reward anticipation in healthy individuals with low, mid, and high levels of schizotypy

A growing body of research has placed the ventral striatum at the center of a network of cerebral regions involved in anticipating rewards in healthy controls. However, little is known about the functional connectivity of the ventral striatum associated with reward anticipation in healthy controls. In addition, few studies have investigated reward anticipation in healthy humans with different levels of schizotypy. Here, we investigated reward anticipation in eighty-four healthy individuals (44 females) recruited based on their schizotypy scores. Participants performed a variant of the Monetary Incentive Delay Task while undergoing event-related fMRI.Participants showed the expected decrease in response times for highly rewarded trials compared to non-rewarded trials. Whole-brain activation analyses replicated previous results, including activity in the ventral and dorsal striatum. Whole-brain psycho-physiological interaction analyses of the left and right ventral striatum revealed increased connectivity during reward anticipation with widespread regions in frontal, parietal and occipital cortex as well as the cerebellum and midbrain. Finally, we found no association between schizotypal personality severity and neural activity and cortico-striatal functional connectivity. In line with the motivational, attentional, and motor functions of rewards, our data reveal multifaceted cortico-striatal networks taking part in reward anticipation in healthy individuals. The ventral striatum is connected to regions of the salience, attentional, motor and visual networks during reward anticipation and thereby in a position to orchestrate optimal goal-directed behavior.

The Effect of Efavirenz on Reward Processing in Asymptomatic People Living with HIV: A Randomized Controlled Trial.

Functional magnetic resonance imaging (fMRI) studies have demonstrated that HIV-infection affects the fronto-striatal network. It has not been examined what impact efavirenz (EFV), an antiretroviral drug notorious for its neurocognitive effects, has on the reward system: a key subcomponent involved in depressive and apathy symptoms. Therefore, this study aims to investigate the effect of EFV on reward processing using a monetary incentive delay (MID) task. In this multicenter randomized controlled trial, asymptomatic adult participants stable on emtricitabine/tenofovirdisoproxil fumarate (FTC/TDF)/EFV were randomly assigned in a 2:1 ratio to switch to FTC/TDF/rilpivirine (RPV) (n = 30) or continue taking FTC/TDF/EFV (n = 13). At baseline and 12 weeks after therapy switch, both groups performed an MID task. Behavior and functional brain activity related to reward anticipation and reward outcome were assessed with blood-oxygen-level-dependent fMRI. Both groups were matched for age, education level, and time since HIV diagnosis and on EFV. At the behavioral level, both groups had faster response times and better response accuracy during rewarding versus nonrewarding trials, with no improvement resulting from switching FTC/TDF/EFV to FTC/TDF/RPV. No significant change in activation related to reward anticipation in the ventral striatum was found after switching therapy. Both groups had significantly higher activation levels over time, consistent with a potential learning effect. Similar activity related to reward outcome in the orbitofrontal cortex was found. Discontinuing FTC/TDF/EFV was not found to improve activity related to reward anticipation in asymptomatic people living with HIV, with similar cortical functioning during reward outcome processing. It is therefore likely that EFV does not affect motivational control. Further research is needed to determine whether EFV affects motivational control in HIV populations with different characteristics.

Dissociating reward sensitivity and negative urgency effects on impulsivity in the five-choice serial reaction time task.

Negative urgency describes the tendency for rash and impulsive behaviour during negative emotional states and has been linked to a number of psychiatric disorders. However, there has been limited research on negative urgency as an explanatory mechanism for impulsivity in experimental animals. Such research has important implications for elucidating the neurobiology of negative urgency and thereby the development of future therapeutic interventions. In this study, we investigated the effects of negative urgency using a partial reinforcement schedule to increase the frequency of non-rewarded (i.e. frustrative) trials in the five-choice serial reaction time task, a widely used task to assess visual attention and impulsivity. Using a Markov chain model to analyse trial-by-trial outcomes we found that premature (i.e. impulsive) responses in the five-choice serial reaction time task were more likely to occur after a non-rewarded trial, and mostly after a previous premature trial. However, contrary to the frustration hypothesis of negative urgency, increasing the probability of reinforcement (p(R)) from p(R) = 0.5 to p(R) = 1 increased the number of premature responses in each session. Micro and macro levels of analyses revealed that impulsivity in the five-choice serial reaction time task is governed by at least two processes, one dependent on the overall level of reinforcement hypothesised to determine the state of behavioural activation, the second dependent on trial-by-trial outcomes consistent with negative urgency effects. These processes may depend on distinct neurobiological mechanisms and have relevance for neuropsychiatric disorders that implicate impulsive behaviours dependent on positive and negative affective states.

Task Learning Over Multi-Day Recording via Internally Rewarded Reinforcement Learning Based Brain Machine Interfaces.

Autonomous brain machine interfaces (BMIs) aim to enable paralyzed people to self-evaluate their movement intention to control external devices. Previous reinforcement learning (RL)-based decoders interpret the mapping between neural activity and movements using the external reward for well-trained subjects, and have not investigated the task learning procedure. The brain has developed a learning mechanism to identify the correct actions that lead to rewards in the new task. This internal guidance can be utilized to replace the external reference to advance BMIs as an autonomous system. In this study, we propose to build an internally rewarded reinforcement learning-based BMI framework using the multi-site recording to demonstrate the autonomous learning ability of the BMI decoder on the new task. We test the model on the neural data collected over multiple days while the rats were learning a new lever discrimination task. The primary motor cortex (M1) and medial prefrontal cortex (mPFC) spikes are interpreted by the proposed RL framework into the discrete lever press actions. The neural activity of the mPFC post the action duration is interpreted as the internal reward information, where a support vector machine is implemented to classify the reward vs. non-reward trials with a high accuracy of 87.5% across subjects. This internal reward is used to replace the external water reward to update the decoder, which is able to adapt to the nonstationary neural activity during subject learning. The multi-cortical recording allows us to take in more cortical recordings as input and uses internal critics to guide the decoder learning. Comparing with the classic decoder using M1 activity as the only input and external guidance, the proposed system with multi-cortical recordings shows a better decoding accuracy. More importantly, our internally rewarded decoder demonstrates the autonomous learning ability on the new task as the decoder successfully addresses the time-variant neural patterns while subjects are learning, and works asymptotically as the subjects' behavioral learning progresses. It reveals the potential of endowing BMIs with autonomous task learning ability in the RL framework.

Evidence that haloperidol impairs learning and motivation scores in a probabilistic task by reducing the reward expectation

Activation of midbrain dopamine neurons in response to positive prediction errors and reward predictive cues is proposed to “energize” reward seeking behaviors and approach responses to places where the reward is expected. In the present study, we tested the effect of the D2-dopamine receptor antagonist haloperidol on response latencies to enter two arms of a Y-maze with different reward probabilities. Adult male Wistar rats were trained to explore the Y-maze with sucrose pellets placed 30% of times at the end of one arm and 70% of times at the opposite arm. Therefore, the reward expectation was different among arms, and was updated in the trials when the reward was omitted. After training, rats received 0.05, 0.10, 0.15 mg/kg haloperidol, or saline 30 min before the test session. In the last, but not in the first trials, haloperidol caused a dose-dependent increase in arm choice latency and response latency. Saline, but not haloperidol, treated rats presented significantly longer response latencies for the 30% compared to the 70% reward probability arm. Haloperidol also caused a dose-dependent decrease in the number of entries in the 70% reward probability arm, increased the number of non-responses, and caused a dose-dependent increase in the number of re-entries in the 30% reward probability arm after non-rewarded trials. Control experiments suggested that haloperidol did not cause motor impairment or satiation, but rather impaired learning and motivation scores by reducing the reward expectation.

Quantifying exploration in reward-based motor learning.

Exploration in reward-based motor learning is observable in experimental data as increased variability. In order to quantify exploration, we compare three methods for estimating other sources of variability: sensorimotor noise. We use a task in which participants could receive stochastic binary reward feedback following a target-directed weight shift. Participants first performed six baseline blocks without feedback, and next twenty blocks alternating with and without feedback. Variability was assessed based on trial-to-trial changes in movement endpoint. We estimated sensorimotor noise by the median squared trial-to-trial change in movement endpoint for trials in which no exploration is expected. We identified three types of such trials: trials in baseline blocks, trials in the blocks without feedback, and rewarded trials in the blocks with feedback. We estimated exploration by the median squared trial-to-trial change following non-rewarded trials minus sensorimotor noise. As expected, variability was larger following non-rewarded trials than following rewarded trials. This indicates that our reward-based weight-shifting task successfully induced exploration. Most importantly, our three estimates of sensorimotor noise differed: the estimate based on rewarded trials was significantly lower than the estimates based on the two types of trials without feedback. Consequently, the estimates of exploration also differed. We conclude that the quantification of exploration depends critically on the type of trials used to estimate sensorimotor noise. We recommend the use of variability following rewarded trials.

Behavioural evidence for colour vision determined by conditioning in the purple mud crab Scylla tranquebarica

Crabs and shrimps (order Decapoda) use colours in various tasks such as foraging and mate choice. Colour vision requires at least two types of photoreceptors with different spectral sensitivities. Previous physiological studies revealed that most crabs including Scylla mud crabs have a single visual receptor system, i.e. they are colour blind. We determined colour vision by means of a behavioural experiment on hatchery-produced and wild-captured purple mud crab Scylla tranquebarica in a roofed hatchery. Adult crabs (8-10 cm carapace width) were subjected to classical conditioning to associate a food reward with a blue or a green stimulus placed among seven shades of grey. The hatchery-produced crabs learnt this task after 14 days of reward training, and thereafter distinguished blue in 27 non-reward trials, and green in 39 non-reward trials. The wild-captured crabs did so after 25 days of reward training, and distinguished green in 49 non-reward trials. These results indicated colour vision in S. tranquebarica. However, the crabs were unable to distinguish blue or green in dim light of 4.4 cd/m2 (which is slightly brighter than full moon light). The high colour vision threshold was attributed to the small optic parameters of the apposition compound eyes of S. tranquebarica.

Modeling Interval Timing by Recurrent Neural Nets.

The purpose of this study was to take a new approach in showing how the central nervous system might encode time at the supra-second level using recurrent neural nets (RNNs). This approach utilizes units with a delayed feedback, whose feedback weight determines the temporal properties of specific neurons in the network architecture. When these feedback neurons are coupled, they form a multilayered dynamical system that can be used to model temporal responses to steps of input in multidimensional systems. The timing network was implemented using separate recurrent “Go” and “No-Go” neural processing units to process an individual stimulus indicating the time of reward availability. Outputs from these distinct units on each time step are converted to a pulse reflecting a weighted sum of the separate Go and No-Go signals. This output pulse then drives an integrator unit, whose feedback weight and input weights shape the pulse distribution. This system was used to model empirical data from rodents performing in an instrumental “peak interval timing” task for two stimuli, Tone and Flash. For each of these stimuli, reward availability was signaled after different times from stimulus onset during training. Rodent performance was assessed on non-rewarded trials, following training, with each stimulus tested individually and simultaneously in a stimulus compound. The associated weights in the Go/No-Go network were trained using experimental data showing the mean distribution of bar press rates across an 80 s period in which a tone stimulus signaled reward after 5 s and a flash stimulus after 30 s from stimulus onset. Different Go/No-Go systems were used for each stimulus, but the weighted output of each fed into a final recurrent integrator unit, whose weights were unmodifiable. The recurrent neural net (RNN) model was implemented using Matlab and Matlab’s machine learning tools were utilized to train the network using the data from non-rewarded trials. The neural net output accurately fit the temporal distribution of tone and flash-initiated bar press data. Furthermore, a “Temporal Averaging” effect was also obtained when the flash and tone stimuli were combined. These results indicated that the system combining tone and flash responses were not superposed as in a linear system, but that there was a non-linearity, which interacted between tone and flash. In order to achieve an accurate fit to the empirical averaging data it was necessary to implement non-linear “saliency functions” that limited the output signal of each stimulus to the final integrator when the other was co-present. The model suggests that the central nervous system encodes timing generation as a dynamical system whose timing properties are embedded in the connection weights of the system. In this way, event timing is coded similar to the way other sensory-motor systems, such as the vestibulo-ocular and optokinetic systems, which combine sensory inputs from the vestibular and visual systems to generate the temporal aspects of compensatory eye movements.

Evidence for motivational enhancement of sign-tracking behavior under reward uncertainty.

Reward uncertainty has been shown to invigorate rather than attenuate cue attraction and responding. For example, a number of findings have shown that partial reinforcement in autoshaping increases response rates to a conditioned stimulus (conditional stimulus) in comparison with continuous reinforcement. However, identifying the nature of this effect remains a topical question. The frustration theory posits that animals are frustrated by reward loss and predicts that enhanced responding results from higher response rates to conditional stimulus presentations that follow nonrewarded trials rather than rewarded trials. In contrast, the incentive hope hypothesis suggests that animals are motivated by possible future rewards and predicts similar response rates after rewarded and nonrewarded trials. Our results, which consist of a reanalysis of previously published data (Hellberg, Levit, & Robinson, 2018), are consistent with the incentive hope hypothesis because no differences were found between trials that follow rewarded or nonrewarded trials, or between trials that follow small or larger amounts of food reward in rats. There was also no evidence for an accumulation of frustration across each training session, with rats instead displaying enhanced yet stable responding from beginning to end. The incentive hope hypothesis is also briefly discussed in relation to the concept of incentive salience. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

"A" for Effort: Rewarding Effortful Retrieval Attempts Improves Learning From General Knowledge Errors in Women.

Previous research has shown that the prospect of attaining a reward can promote task-engagement, up-regulate attention toward reward-relevant information, and facilitate enhanced encoding of new information into declarative memory. However, past research on reward-based enhancement of declarative memory has focused primarily on paradigms in which rewards are contingent upon accurate responses. Yet, findings from test-enhanced learning show that making errors can also be useful for learning if those errors represent effortful retrieval attempts and are followed by corrective feedback. Here, we used a challenging general knowledge task to examine the effects of explicitly rewarding retrieval effort, defined as a semantically plausible answer to a question (referenced to a semantic knowledge database www.mangelslab.org/bknorms), regardless of response accuracy. In particular, we asked whether intermittent rewards following effortful incorrect responses facilitated learning from corrective feedback as measured by incidental learning outcomes on a 24–48 h delayed retest. Given that effort-contingent extrinsic rewards represent the intersection between an internal locus of control and competency, we compared participants in this “Effort” group to three other groups in a between-subjects design: a Luck group that framed rewards as related to participant-chosen lottery numbers (reward with internal control, not competence-based), a random Award group that framed rewards as computer generated (no control, not competence-based), and a Control group with no reward, but matched on all other task features. Both men and women in the Effort group showed increased self-reports of concentration and positive feelings following the receipt of rewards, as well as subjective effort on the retest, compared to the Control group. However, only women additionally exhibited performance benefits of effort framing on error correction. These benefits were found for both rewarded and non-rewarded trials, but only for correction of low confidence errors, suggesting that effort-contingent rewards produced task-level changes in motivation to learn less familiar information in women, rather than trial-level influences in encoding or consolidation. The Luck and Award groups did not demonstrate significant motivational or behavioral benefits for either gender. These results suggest that both reward context and gender are important factors contributing to the effectiveness of rewards as tools to enhance learning from errors.

Regionally distinct phasic dopamine release patterns in the striatum during reversal learning

Striatal dopamine (DA) plays a central role in reward-related learning and behavioral adaptation to changing environments. Recent studies suggest that rather than being broadcast as a uniform signal throughout the entire region, DA release dynamics diverge between different striatal regions. In a previous study, we showed that phasic DA release patterns in the ventromedial striatum (VMS) rapidly adapt during reversal learning. However, it is unknown how DA dynamics in the dorsolateral striatum (DLS) are modulated during such adaptive behavior. Here, we used fast-scan cyclic voltammetry to measure phasic DA release in the DLS during spatial reversal learning. In the DLS, we observed minor DA release after the onset of a visual cue signaling reward availability, followed by more pronounced DA release during more proximal reward cues (e.g., lever extension) and execution of the operant response (i.e., lever press), both in rewarded and non-rewarded trials. These release dynamics (minor DA after onset of the predictive visual cue, prominent DA during the operant response) did not change significantly during or following a reversal of response-reward contingencies. Notably, the DA increase to the lever press did not reflect a general signal related to the initiation of any motivated motor response, as we did not observe DA release when rats initiated nose pokes into the food receptacle during inter-trial intervals. This suggests that DA release in the DLS occurs selectively during the initiation and execution of a learned operant response.Together with our previous results obtained in the VMS, these findings reveal distinct phasic DA release patterns during adaptation of established behavior in DLS and VMS. The VMS DA signal, which is highly sensitive to reversal of response-reward contingences, may provide a teaching signal to guide reward-related learning and facilitate behavioral adaptation, whereas DLS DA may reflect a ‘response execution signal’ largely independent of outcome, that may be involved in initiation and energizing of operant behavior.

Classifier Performance in Primary Somatosensory Cortex Towards Implementation of a Reinforcement Learning Based Brain Machine Interface.

Increasingly accurate control of prosthetic limbs has been made possible by a series of advancements in brain machine interface (BMI) control theory. One promising control technique for future BMI applications is reinforcement learning (RL). RL based BMIs require a reinforcing signal to inform the controller whether or not a given movement was intended by the user. This signal has been shown to exist in cortical structures simultaneously used for BMI control. This work evaluates the ability of several common classifiers to detect impending reward delivery within primary somatosensory (S1) cortex during a grip force match to sample task performed by a nonhuman primate. The accuracy of these classifiers was further evaluated over a range of conditions to identify parameters that provide maximum classification accuracy. S1 cortex was found to provide highly accurate classification of the reinforcement signal across many classifiers and a wide variety of data input parameters. The classification accuracy in S1 cortex between rewarding and non-rewarding trials was apparent when the animal was expecting an impending delivery or an impending withholding of reward following trial completion. The high accuracy of classification in S1 cortex can be used to adapt an RL based BMI towards a user's intent. Real-time implementation of these classifiers in an RL based BMI could be used to adapt control of a prosthesis dynamically to match the intent of its user.

Habit formation coincides with shifts in reinforcement representations in the sensorimotor striatum.

Evaluating outcomes of behavior is a central function of the striatum. In circuits engaging the dorsomedial striatum, sensitivity to goal value is accentuated during learning, whereas outcome sensitivity is thought to be minimal in the dorsolateral striatum and its habit-related corticostriatal circuits. However, a distinct population of projection neurons in the dorsolateral striatum exhibits selective sensitivity to rewards. Here, we evaluated the outcome-related signaling in such neurons as rats performed an instructional T-maze task for two rewards. As the rats formed maze-running habits and then changed behavior after reward devaluation, we detected outcome-related spike activity in 116 units out of 1,479 recorded units. During initial training, nearly equal numbers of these units fired preferentially either after rewarded runs or after unrewarded runs, and the majority were responsive at only one of two reward locations. With overtraining, as habits formed, firing in nonrewarded trials almost disappeared, and reward-specific firing declined. Thus error-related signaling was lost, and reward signaling became generalized. Following reward devaluation, in an extinction test, postgoal activity was nearly undetectable, despite accurate running. Strikingly, when rewards were then returned, postgoal activity reappeared and recapitulated the original early response pattern, with nearly equal numbers responding to rewarded and unrewarded runs and to single rewards. These findings demonstrate that outcome evaluation in the dorsolateral striatum is highly plastic and tracks stages of behavioral exploration and exploitation. These signals could be a new target for understanding compulsive behaviors that involve changes to dorsal striatum function.

Reward value enhances post-decision error-related activity in the cingulate cortex

By saying “Anyone who has never made a mistake has never tried anything new”, Albert Einstein himself allegedly implied that the making and processing of errors are essential for behavioral adaption to a new or changing environment. These essential error-related cognitive and neural processes are likely influenced by reward value. However, previous studies have not dissociated accuracy and value and so the distinct effect of reward on error processing in the brain remained unknown. Therefore, we set out to investigate this at various points in decision-making. We used functional magnetic resonance imaging to scan participants while they completed a random dot motion discrimination task where reward and non-reward were associated with stimuli via classical conditioning. Pre-error activity was found in the medial frontal cortex prior to response but this was not related to reward value. At response time, error-related activity was found to be significantly greater in reward than non-reward trials in the midcingulate cortex. Finally at outcome time, error-related activity was found in the anterior cingulate cortex in non-reward trials. These results show that reward value enhances post-decision but not pre-decision error-related activities and these results therefore have implications for theories of error correction and confidence.

Effects of affective arousal on choice behavior, reward prediction errors, and feedback-related negativities in human reward-based decision making

Emotional experience has a pervasive impact on choice behavior, yet the underlying mechanism remains unclear. Introducing facial-expression primes into a probabilistic learning task, we investigated how affective arousal regulates reward-related choice based on behavioral, model fitting, and feedback-related negativity (FRN) data. Sixty-six paid subjects were randomly assigned to the Neutral-Neutral (NN), Angry-Neutral (AN), and Happy-Neutral (HN) groups. A total of 960 trials were conducted. Subjects in each group were randomly exposed to half trials of the pre-determined emotional faces and another half of the neutral faces before choosing between two cards drawn from two decks with different assigned reward probabilities. Trial-by-trial data were fit with a standard reinforcement learning model using the Bayesian estimation approach. The temporal dynamics of brain activity were simultaneously recorded and analyzed using event-related potentials. Our analyses revealed that subjects in the NN group gained more reward values than those in the other two groups; they also exhibited comparatively differential estimated model-parameter values for reward prediction errors. Computing the difference wave of FRNs in reward vs. non-reward trials, we found that, compared to the NN group, subjects in the AN and HN groups had larger “General” FRNs (i.e., FRNs in no-reward trials minus FRNs in reward trials) and “Expected” FRNs (i.e., FRNs in expected reward-omission trials minus FRNs in expected reward-delivery trials), indicating an interruption in predicting reward. Further, both AN and HN groups appeared to be more sensitive to negative outcomes than the NN group. Collectively, our study suggests that affective arousal negatively regulates reward-related choice, probably through overweighting with negative feedback.

Toward an autonomous brain machine interface: integrating sensorimotor reward modulation and reinforcement learning.

For decades, neurophysiologists have worked on elucidating the function of the cortical sensorimotor control system from the standpoint of kinematics or dynamics. Recently, computational neuroscientists have developed models that can emulate changes seen in the primary motor cortex during learning. However, these simulations rely on the existence of a reward-like signal in the primary sensorimotor cortex. Reward modulation of the primary sensorimotor cortex has yet to be characterized at the level of neural units. Here we demonstrate that single units/multiunits and local field potentials in the primary motor (M1) cortex of nonhuman primates (Macaca radiata) are modulated by reward expectation during reaching movements and that this modulation is present even while subjects passively view cursor motions that are predictive of either reward or nonreward. After establishing this reward modulation, we set out to determine whether we could correctly classify rewarding versus nonrewarding trials, on a moment-to-moment basis. This reward information could then be used in collaboration with reinforcement learning principles toward an autonomous brain-machine interface. The autonomous brain-machine interface would use M1 for both decoding movement intention and extraction of reward expectation information as evaluative feedback, which would then update the decoding algorithm as necessary. In the work presented here, we show that this, in theory, is possible.

Pigeons rank-order responses to temporally sequential stimuli

We explored pigeons' ability to learn a particular sequence of stimuli in which the durations of each stimulus varied among trials, where the first response at the end of the sequence was reinforced. In Experiment 1A, we found that pigeons failed to use the whole sequence of three stimuli to predict food reinforcement, and instead responded only to the third, "rewarded" stimulus. When rewarded (1-2-3) and nonrewarded (2-1-3) sequences were used in a go/no-go procedure in Experiment 1B, however, pigeons showed a tendency to rank-order responding, with higher response rates to the second than to the first stimulus, as well as lower response rates to the third stimulus on nonrewarded-sequence trials. In Experiment 2, pigeons showed rudimentary rank-ordering of five stimuli in sequence, with lower responding to the final stimulus on nonrewarded trials, even when the sequence presented differed from the rewarded sequence only in a reversal of the second and third stimuli. Pigeons were capable of using ordinal information in a temporal task, but only when that information was easily discriminable and led to explicit consequences (i.e., rewarded vs. nonrewarded sequences).

Interval timing behavior in Pallas’s long-tongued bat (Glossophaga soricina).

Timing behavior in animals and its underlying mechanisms have been investigated extensively in the peak procedure, a variant of fixed interval procedures. In such experiments, individuals typically start responding with high frequency after an initial inactive time interval and continue their responses after peak time if rewards are omitted. This begs the so far unexplored question as to how timing behavior is influenced when such continuous responses are suppressed. Here, we present results from a nectar-feeding bat species, Glossophaga soricina, that was tested in a modified version of the peak procedure at three fixed time intervals (5 s, 11 s, 20 s). In contrast to standard peak procedures we imposed metabolic costs on individual responses which effectively suppressed trains of rapid responses during trials. Under this manipulation, bats' aggregated responses showed clear peaks around the peak time in the 5-s and 11-s schedules. Bats' responses in the 20-s schedule, however, did not peak around the fixed interval time. Crucially, an analysis of time intervals between successive revisits in all schedules revealed that bats revisited feeders at accurately timed intervals in all three conditions. The individual within trial behavioral responses showed clear oscillatory patterns throughout nonrewarded trials. These findings follow predictions from mechanistic timing models, like the striatal beat frequency model, and are discussed with regard to these models.

Interval timing under variations in the relative validity of temporal cues.

Two groups of pigeons were trained to respond on a white center key to a fixed-interval, 60-s schedule of reinforcement signaled by the onset of a side-key cue (S+ training). In additional training sessions, S+ trials alternated between S- trials in which a different side-key cue signaled nonreinforcement after 60 s (S+/S- training). For one group, S+/S- training sessions followed S+ training, and for the other group, S+/S- training preceded S+ training. Peak-time curves obtained from extended nonrewarded probe trials inserted among training trials showed loss of control by time during S+/S- training relative to S+ training. A follow-up experiment showed that this result was not caused by a difference in probability of reinforcement. We suggest that attention to time was weakened by the introduction of visual cues that were more valid predictors of trial outcomes.

Dopaminergic involvement in effort-based but not impulsive reward processing in smokers

BackgroundA reduction in reward responsivity and an increase in temporal discounting of rewards are both evident in smokers during acute abstinence compared to satiation. However, it is not yet known whether these processes can be modulated pharmacologically in smokers, other than with nicotine or tobacco. MethodsA double-blind placebo controlled crossover design assessed the effects of 0.5mg pramipexole, a dopamine D2/D3 agonist, in smokers following 2h of abstinence. Reward responsivity was measured using an effort-based card sorting task. Temporal discounting of monetary reward was assessed using Area Under the Curve (AUC) analysis, and affective and subjective effects were indexed. ResultsOn placebo, smokers showed an equivalent speed of card sorting when a financial incentive was provided compared to when it was not. Conversely, more cards were sorted under rewarded compared to non-rewarded trials after pramipexole, indicating an improvement in reward responsivity. Temporal discounting of monetary reward was not affected by pramipexole. Drug treatment also decreased positive affect and increased drowsiness. ConclusionsA single dose of pramipexole can enhance effort-based reward responsivity, but does not alter temporal discounting in smokers. These findings highlight pharmacological correlates of reward processing deficits in nicotine dependence and offer potential targets for their treatment.