Changes In Reinforcer Value Research Articles

49502 CHANGES IN DEPENDENCE, WITHDRAWAL, AND CRAVING AMONG ADULT SMOKERS WHO SWITCH TO NICOTINE SALT POD-BASED E-CIGARETTES

ABSTRACT IMPACT: This research suggests that African American and Latinx smokers who bear a disproportionate burden of tobacco-related harms are able to switch to e-cigarettes that present reduced harm to the user due to their similar reinforcement profile to cigarettes. OBJECTIVES/GOALS: Complete switching from combustible to electronic cigarettes (ECs) reduces harm to the user. For ECs to be a viable substitute, they need to be rewarding enough for regular use, indicated by factors such as craving and dependence (reinforcement value). Little is known about short-term changes in reinforcement value across trajectories of EC use. METHODS/STUDY POPULATION: Participants were randomized 2:1 to switch to a nicotine salt pod system EC or continue smoking (assessment-only control) in a 6-week trial. 114 African American (n=60) and Latinx (n=54) smokers were randomized to receive ECs and are included in the current investigation. At week 6, participants were classified by use trajectory: exclusive smokers (n=16), exclusive EC users (n=32), or dual users (n=66). Participants reported on their EC, cigarette, and total nicotine dependence (cigarette + EC dependence), cigarette and EC use, and nicotine craving and withdrawal at baseline and week 6. Cotinine and exhaled carbon monoxide were assessed at baseline and week 6. RESULTS/ANTICIPATED RESULTS: Participants who completely switched from smoking to ECs (exclusive EC users) and those that partially switched (dual users), maintained cotinine levels (ps>.05) and showed reductions in cigarette dependence and withdrawal (ps<.01). However, exclusive EC users showed no significant changes in total nicotine dependence from baseline to week 6 (p=.123), while dual users showed increased total nicotine dependence (p<.001). Dual users displayed similar levels of EC dependence as exclusive EC users but a lesser reduction in cigarette dependence. Exclusive EC users and dual users showed reductions in craving and withdrawal from baseline to week 6. DISCUSSION/SIGNIFICANCE OF FINDINGS: This study is among the first to prospectively examine changes in dependence, craving, and withdrawal among an understudied sample of smokers making a switch attempt. Smokers who completely switch to ECs maintain nicotine levels and dependence, suggesting that they have a similar reinforcement profile to cigarettes and facilitate switching.

Contextual control of instrumental actions and habits.

After a relatively small amount of training, instrumental behavior is thought to be an action under the control of the motivational status of its goal or reinforcer. After more extended training, behavior can become habitual and insensitive to changes in reinforcer value. Recently, instrumental responding has been shown to weaken when tested outside of the training context. The present experiments compared the sensitivity of instrumental responding in rats with a context switch after training procedures that might differentially generate actions or habits. In Experiment 1, lever pressing was decremented in a new context after either short, medium, or long periods of training on either random-ratio or yoked random-interval reinforcement schedules. Experiment 2 found that more minimally trained responding was also sensitive to a context switch. Moreover, Experiment 3 showed that when the goal-directed component of responding was removed by devaluing the reinforcer, the residual responding that remained was still sensitive to the change of context. Goal-directed responding, in contrast, transferred across contexts. Experiment 4 then found that after extensive training, a habit that was insensitive to reinforcer devaluation was still decremented by a context switch. Overall, the results suggest that a context switch primarily influences instrumental habit rather than action. In addition, even a response that has received relatively minimal training may have a habit component that is insensitive to reinforcer devaluation but sensitive to the effects of a context switch.

A quantitative analysis of the reward-enhancing effects of nicotine using reinforcer demand.

Reward enhancement by nicotine has been suggested as an important phenomenon contributing toward tobacco abuse and dependence. Reinforcement value is a multifaceted construct not fully represented by any single measure of response strength. The present study evaluated the changes in the reinforcement value of a visual stimulus in 16 male Sprague-Dawley rats using the reinforcer demand technique proposed by Hursh and Silberberg. The different parameters of the model have been shown to represent differing facets of reinforcement value, including intensity, perseverance, and sensitivity to changes in response cost. Rats lever-pressed for 1-min presentations of a compound visual stimulus over blocks of 10 sessions across a range of response requirements (fixed ratio 1, 2, 4, 8, 14, 22, 32). Nicotine (0.4 mg/kg, base) or saline was administered 5 min before each session. Estimates from the demand model were calculated between nicotine and saline administration conditions within subjects and changes in reinforcement value were assessed as differences in Q0, Pmax, Omax, and essential value. Nicotine administration increased operant responding across the entire range of reinforcement schedules tested, and uniformly affected model parameter estimates in a manner suggesting increased reinforcement value of the visual stimulus.

Neurotoxic Lesions of the Medial Mediodorsal Nucleus of the Thalamus Disrupt Reinforcer Devaluation Effects in Rhesus Monkeys

The mediodorsal thalamus is a major input to the prefrontal cortex and is thought to modulate cognitive functions of the prefrontal cortex. Damage to the medial, magnocellular part of the mediodorsal thalamus (MDmc) impairs cognitive functions dependent on prefrontal cortex, including memory. The contribution of MDmc to other aspects of cognition dependent on prefrontal cortex has not been determined. The ability of monkeys to adjust their choice behavior in response to changes in reinforcer value, a capacity impaired by lesions of orbital prefrontal cortex, can be tested in a reinforcer devaluation paradigm. In the present study, rhesus monkeys with bilateral neurotoxic MDmc lesions were tested in the devaluation procedure. Monkeys learned visual discrimination problems in which each rewarded object is reliably paired with one of two different food rewards and then were given choices between pairs of rewarded objects, one associated with each food. Selective satiation of one of the food rewards reduces choices of objects associated with that food in normal monkeys. Monkeys with bilateral neurotoxic lesions of MDmc learned concurrently presented visual discrimination problems as quickly as unoperated control monkeys but showed impaired reinforcer devaluation effects. This finding suggests that the neural circuitry for control of behavioral choice by changes in reinforcer value includes MDmc.

The contribution of ventrolateral and dorsolateral prefrontal cortex to response reversal

Studies investigating response reversal consistently implicate regions of medial and lateral prefrontal cortex when reinforcement contingencies change. However, it is unclear from these studies how these regions give rise to the individual components of response reversal, such as reinforcement value encoding, response inhibition, and response change. Here we report a novel instrumental learning task designed to determine whether regions implicated in processing reversal errors are uniquely involved in this process, or whether they play a more general role in representing response competition, reinforcement value, or punishment value in the absence of demands for response change. In line with previous findings, reversal errors activated orbitofrontal cortex, dorsomedial prefrontal cortex, ventrolateral prefrontal cortex, caudate, and dorsolateral prefrontal cortex. These regions also showed increased activity to errors in the absence of contingency changes. In addition, ventrolateral PFC, caudate, and dorsolateral PFC each exhibited increased activity following correct reversals. Activity in these regions was not significantly modulated by changes in reinforcement value that were not sufficient to make an alternative response advantageous. These data do not support punishment-processing or prepotent response inhibition accounts of ventrolateral prefrontal cortex function. Instead, they support recent conceptualizations of ventrolateral prefrontal cortex function that implicate this region in resolving response competition by manipulating the representation of either motor response options, or object features. These data also suggest that dorsolateral prefrontal cortex plays a role in reversal learning, probably through top down attentional control of object or reinforcement features when task demands increase.

The Measurement and Functional Properties of Reinforcer Value in Single-Alternative Responding: A Test of Linear System Theory

Matching theory and linear system theory make different predictions about how the y-asymptote, k, of Herrnstein’s (1970) hyperbola varies with reinforcer value. The present experiment tested both these predictions and linear system theory’s account of how reinforcement rate and reinforcer value jointly govern response rate. Eight rats served as subjects. In Phase 1, 3 sucrose pellets of different concentration (5%, 75%, and 95%) were alternately paired with a fourth concentration (50%) on concurrent VI VI schedules. By fitting linear system theory’s equation for concurrent choice, measures of reinforcer value were assigned to each concentration for each subject. The reinforcer values for the 50%, 75%, and 95% were roughly equivalent, and the reinforcer values for the 5% concentration were lower. In Phase 2 subjects were exposed to a series of single-alternative VI schedules (8, 17, 55, 150, and 250 sec) and each sucrose concentration was studied separately, with a 25% pellet added. Herrnstein’s equation was fitted to response rate as a function of reinforcement rate for each concentration. The asymptote, k, was roughly equivalent under the 50%, 75%, and 95% pellets, but was lower for the 5% and 25% pellets. The measures of reinforcer value from Phase 1 suggest that the equivalent ks in Phase 2 were the result of equivalent reinforcer values. Matching theory predicts that k should remain constant across changes in reinforcer value. The value of k varied with reinforcer value, and the form of variability in k supported linear system theory’s prediction of a positively accelerating function. Linear system theory’s prediction about how reinforcement rate and reinforcer value jointly govern response rate was also supported. The results contradict matching theory’s account and support linear system theory’s account of single-alternative responding.

Preserved Sensitivity to Outcome Value after Lesions of the Basolateral Amygdala

Recent work (Blundell et al., 2001; Balleine et al., 2003) has suggested that the basolateral region of the amygdala (BLA) is important in the representation of the sensory and incentive aspects of motivationally significant events. In common with other theories of function of the BLA, this predicts that lesions of the BLA will interfere with reinforcer devaluation after appetitive Pavlovian or instrumental conditioning. However, this hypothesis also predicts that BLA lesions will be without effect on postconditioning changes in reinforcer value if initial learning is only about the sensory aspects of otherwise neutral events. This interpretation is supported by evidence for significant detrimental effects of BLA lesions on reinforcer devaluation in a Pavlovian autoshaping procedure, but no effect of postconditioning devaluation using a sensory preconditioning procedure. These results demonstrate that animals with BLA lesions can remain sensitive to post-training changes in the motivational value of outcomes.

Dynamic changes in reinforcer value: Some misconceptions and why you should care.

Abstract The effectiveness of a reinforcer in maintaining behavior (its value) changes systematically with successive deliveries of that reinforcer. Some misconceptions have impeded our understanding of these changes. The misconceptions include the idea that the changes never occur or occur only when large reinforcers are used; that they always occur; that they are always bitonic in form; that they have no theoretical implications; and that they are produced by to the reinforcer. A precise characterization of the factors that alter the effectiveness of a reinforcer is essential for the theoretical understanding of operant behavior and for the use of operant techniques in practice. ********** For many years, operant psychologists have believed that reinforcers lose their effectiveness (i.e., their value) as they are repeatedly presented (e.g., Reese & Hogenson, 1962; Skinner, 1932). This finding has potentially important theoretical and practical implications. Changes in reinforcer value are theoretically important because they are not anticipated by most current theories. As will be argued, the changes may also contribute to the theoretical explanation of a number of operant phenomena, such as behavioral contrast and spontaneous recovery (see Within-session Changes are not Theoretically Important). Dynamic changes in reinforcer value are practically important because operant techniques are often used to strengthen or weaken behaviors. To do this effectively, contingency managers must maintain the effectiveness of their reinforcer or punisher as long as possible. In spite of their potential importance, dynamic changes in reinforcer value were never subjected to an experimental analysis. Instead, they were labeled, satiation (e.g., Reese & Hogenson, 1962). This was regrettable because the only evidence that reinforcers lost their effectiveness with repeated presentation was that operant response rates declined after a large number of reinforcers had been delivered. Factors other than (e.g., fatigue) could have contributed to these declines, but those factors were never ruled out by experimental test. Additionally, even if changes in reinforcer value did produce the declines in response rates, factors other than (e.g., habituation) could have produced those changes in reinforcer value. The study of dynamic changes in reinforcer value was ignored until the rediscovery of the fact that rates of operant responding may not be constant within experimental sessions even when the conditions of reinforcement are constant across the session. Instead, rate of responding often increases, decreases, or increases and then decreases within sessions (e.g., McSweeney, 1992; McSweeney, Hatfield,& Allen, 1990). Subsequent research confirmed that these within-session changes in responding are produced primarily by dynamic changes in the value of the reinforcer For example, probe preference tests show that reinforcer value changes systematically within sessions (McSweeney, Weatherly, & Swindell, 1996a). In addition, several competing explanations for within-session response patterns were ruled out. Rejected explanations include: recovery from handling (McSweeney & Johnson, 1994), anticipation of events that follow the session (e.g., feeding, McSweeney, Weatherly, & Swindell, 1995), changes in a general motivational state (e.g., arousal, McSweeney, Swindell, & Weatherly, 1996a; 1996c), changes in interference from adjunctive behaviors (McSweeney, Swindell, & Weatherly, 1996a) or exploration (Roll & McSweeney, 1997), changes in factors produced by the act of responding, such as muscular warm-up or fatigue (McSweeney, Weatherly, & Roll, 1995; McSweeney, Weatherly, Roll, & Swindell, 1995; Melville, Rybiski, & Kamrani, 1996; Weatherly, McSweeney, & Swindell, 1995), and changes in attention to the task defined in several ways (McSweeney, Roll, & Weatherly, 1994; McSweeney, Weatherly, & Swindell, 1996c; Melville & Weatherly, 1996). …

Multiple variable interval schedule behaviour in humans: effects of ethanol, mood, and reinforcer size on responding maintained by monetary reinforcement.

Ethanol is an effective reinforcer but, in common with other drugs of abuse, it may derive some of its reinforcing properties from the effects it has on behaviour maintained by other reinforcers. However, any assessment of ethanol's hypothesized effect on behaviour maintained by other reinforcers must take into account the fact that ethanol may have multiple mechanisms of action. In order to address this problem the experiments reported herein used a procedure based upon Herrnstein's Matching Law which allowed joint assessment of subjects' motor capacity and reinforcer sensitivity. The effect of ethanol (0, 0.3, and 0.6 g/kg) on motor capacity and reinforcer sensitivity was assessed by studying behaviour maintained by monetary reinforcement. In the first experiment the procedure was validated by showing that the behaviour of subjects was sensitive to changes in reinforcer value and in the second experiment 0.6 g/kg ethanol reduced motor capacity but did not affect reinforcer sensitivity. As a secondary hypothesis we also studied the effect of mood on reinforcer sensitivity and motor capacity. It was found that lower mood scores (lower hedonic tone) were associated with reduced reinforcer sensitivity and that male subjects showed higher motor capacity than females. However, there was also a mood by sex interaction, which indicated that higher motor capacity in males was only found in the presence of lower mood scores. The results are discussed in relation to the mechanisms of ethanol's dopaminergic effects, interactions between ethanol and other drugs of abuse, and the changes in reinforcer sensitivity which are thought to occur in depression.

Reinforcer value may change within experimental sessions

Large and systematic changes in response rates often occur within sessions during operant conditioning procedures. In the present experiment, we asked whether the value of the reinforcer that supports responding also changes within sessions. Pigeons pecked a key for mixed grain available throughout the session. Occasionally, wheat was also provided for pecking a second key. The ratio of the rates of responding for mixed grain and wheat, a frequently used measure of relative reinforcer value, changed significantly within sessions when mixed grain was provided at high, but not at low, rates. Habituation to the reinforcer provides the most likely explanation for these changes in reinforcer value. Eventually, habituation may provide a unified explanation for the within-session changes in behavior that occur when many species of subjects respond on a wide variety of tasks.

DRL performance, extinction, and secondary reinforcement: role of appetitive value of food in mice with septal lesions.

Normal mice and mice with septal lesions were trained on a differential-reinforcement-for-low-rates-of-responding (DRL 8-sec) schedule for the reinforcement varying in incentive value. Dilution of diet increased the number of reinforcements received by mice with septal lesions. In Experiments 2 and 3 the effects of septal lesions on resistance to extinction and continuous reinforcement training and the strength of secondary reinforcement were investigated. Changes in reinforcement value modified the septal lesion effects in both cases. All three experiments demonstrated the alteration of the septal lesion effect through a change in the appetitive value of the reinforcement. The results suggest that one of the mechanisms by which septal lesions impair DRL performance is an enhancement of reinforcing properties of food.