Variable-interval 30-s Research Articles

Progesterone attenuates impulsive action in a Go/No-Go task for sucrose pellets in female and male rats

Impulsivity, or a tendency to act without anticipation of future consequences, is associated with drug abuse. Impulsivity is typically separated into two main measures, impulsive action and impulsive choice. Given the association of impulsivity and drug abuse, treatments that reduce impulsivity have been proposed as an effective method for countering drug addiction. Progesterone has emerged as a promising treatment, as it is associated with decreased addiction-related behaviors and impulsive action. The goal of the present study was to determine the effects of progesterone (PRO) on impulsive action for food: a Go/No-Go task. Female and male rats responded for sucrose pellets during a Go component when lever pressing was reinforced on a variable-interval 30-s schedule. During the alternate No-Go component, withholding a lever press was reinforced on a differential reinforcement of other (DRO) behavior 30-s schedule, where a lever press reset the DRO timer. Impulsive action was operationally defined as the inability to withhold a response during the No-Go component (i.e. the number of DRO resets). Once Go/No-Go behavior was stable, responding between rats treated with PRO (0.5mg/kg) or vehicle was examined. Progesterone significantly decreased the total number of DRO resets in both males and females, but it did not affect VI responding for sucrose pellets. This suggests that PRO decreases motor impulsivity for sucrose pellets without affecting motivation for food. Thus, PRO may reduce motor impulsivity, a behavior underlying drug addiction.

Resistance to change and resurgence in humans engaging in a computer task

The relation between persistence, as measured by resistance to change, and resurgence has been examined with nonhuman animals but not systematically with humans. The present study examined persistence and resurgence with undergraduate students engaging in a computer task for points exchangeable for money. In Phase 1, a target response was maintained on a multiple variable-interval (VI) 15-s (Rich) VI 60-s (Lean) schedule of reinforcement. In Phase 2, the target response was extinguished while an alternative response was reinforced at equal rates in both schedule components. In Phase 3, the target and the alternative responses were extinguished. In an additional test of persistence (Phase 4), target responding was reestablished as in Phase 1 and then disrupted by access to videos in both schedule components. In Phases 2 and 4, target responding was more persistent in the Rich than in the Lean component. Also, resurgence generally was greater in the Rich than in the Lean component in Phase 3. The present findings with humans extend the generality of those obtained with nonhuman animals showing that higher reinforcement rates produce both greater persistence and resurgence, and suggest that common processes underlie response persistence and relapse.

Wheel-running reinforcement in free-feeding and food-deprived rats

Rats experiencing sessions of 30min free access to wheel running were assigned to ad-lib and food-deprived groups, and given additional sessions of free wheel activity. Subsequently, both ad-lib and deprived rats lever pressed for 60s of wheel running on fixed ratio (FR) 1, variable ratio (VR) 3, VR 5, and VR 10 schedules, and on a response-initiated variable interval (VI) 30s schedule. Finally, the ad-lib rats were switched to food deprivation and the food-deprived rats were switched to free food, as rats continued responding on the response-initiated VI 30-s schedule. Wheel running functioned as reinforcement for both ad-lib and food-deprived rats. Food-deprived rats, however, ran faster and had higher overall lever-pressing rates than free-feeding rats. On the VR schedules, wheel-running rates positively correlated with local and overall lever pressing rates for deprived, but not ad-lib rats. On the response-initiated VI 30s schedule, wheel-running rates and lever-pressing rates changed for ad-lib rats switched to food deprivation, but not for food-deprived rats switched to free-feeding. The overall pattern of results suggested different sources of control for wheel running: intrinsic motivation, contingencies of automatic reinforcement, and food-restricted wheel running. An implication is that generalizations about operant responding for wheel running in food-deprived rats may not extend to wheel running and operant responding of free-feeding animals.

Mechanisms of impulsive choice: III. The role of reward processes

Two experiments examined the relationship between reward processing and impulsive choice. In Experiment 1, rats chose between a smaller-sooner (SS) reward (1 pellet, 10s) and a larger-later (LL) reward (1, 2, and 4 pellets, 30s). The rats then experienced concurrent variable-interval 30-s schedules with variations in reward magnitude to evaluate reward magnitude discrimination. LL choice behavior positively correlated with reward magnitude discrimination. In Experiment 2, rats chose between an SS reward (1 pellet, 10s) and an LL reward (2 and 4 pellets, 30s). The rats then received either a reward intervention which consisted of concurrent fixed-ratio schedules associated with different magnitudes to improve their reward magnitude discrimination, or a control task. All rats then experienced a post-intervention impulsive choice task followed by a reward magnitude discrimination task to assess intervention efficacy. The rats that received the intervention exhibited increases in post-intervention LL choice behavior, and made more responses for larger-reward magnitudes in the reward magnitude discrimination task, suggesting that the intervention heightened sensitivities to reward magnitude. The results suggest that reward magnitude discrimination plays a key role in individual differences in impulsive choice, and could be a potential target for further intervention developments.

Sensitivity to pre- and post-reinforcer delays in self-control choice

Rats were exposed to a concurrent-chains schedule in which reinforcer amounts, pre-reinforcer and post-reinforcer delays were varied between alternatives. Two independent variable-interval 60-s schedules were in effect during the initial links. Pre- and post-reinforcer delays were defined by fixed-time schedules. In one condition, rats chose between a larger reinforcer delayed by 0, 40, 80, or 120-s and an immediate smaller reinforcer. In the other condition, rats chose between an immediate larger reinforcer terminated with 0, 40, 80, or 120-s post-reinforcer delays and an immediate smaller reinforcer terminated with no post-reinforcer delay. In both conditions, choice was predicted from overall reinforcement density (i.e. reinforcer amount divided by total trial time). The results indicate that the preference for the larger reinforcer systematically decreased as both pre- and post-reinforcer delays increased. Although the effect of pre-reinforcer delay was greater than that of post-reinforcer delay, post-reinforcer delay itself influenced rats' choices in the present self-control situation. In conclusion, the present study demonstrated the limitation of local reinforcement density model of the matching law and all time components that constitute a choice alternative need to be weighted and incorporated into the general model of choice behavior.

An evaluation of resistance to change with unconditioned and conditioned reinforcers.

Several reinforcer-related variables influence a response's resistance to change (Nevin, 1974). Reinforcer type (i.e., conditioned or unconditioned) is a reinforcer-related variable that has not been studied with humans but may have clinical implications. In Experiment 1, we identified unconditioned and conditioned reinforcers of equal preference. In Experiments 2, 3, and 4, we reinforced participants' behavior during a baseline phase using a multiple variable-interval (VI) 30-s VI 30-s schedule with either conditioned (i.e., token) or unconditioned (i.e., food; one type of reinforcement in each component) reinforcement. After equal reinforcement rates across components, we introduced a disruptor. Results of Experiments 2 and 3 showed that behaviors were more resistant to extinction and distraction, respectively, with conditioned than with unconditioned reinforcers. Results of Experiment 4, however, showed that when prefeeding disrupted responding, behaviors were more resistant to change with unconditioned reinforcers than with conditioned reinforcers.

Extra (non-contingent) food does not affect behavioral tolerance to d-amphetamine's rate-decreasing effects

Tolerance refers to the diminished effect of a drug following its repeated administration such that a larger dose is needed to obtain the initial effect. Tolerance to a drug's effects on operant behavior is more likely to develop when initial drug administration results in a loss of reinforcement. It remains unknown how offsetting loss of reinforcement influences the development of tolerance. Providing extra (non-contingent) food pellets was hypothesized to impede the development of tolerance to effects of a repeatedly administered dose of d-amphetamine that reduced the number of food pellets earned by rats. A multiple schedule with two variable-interval (VI) 60-s components resulting in food delivery was used to maintain lever pressing. Extra food pellets were provided in one of those components according to a variable-time (VT) 120-s or a VT 30-s schedule for separate groups of rats (n=6 for each group). Effects of d-amphetamine (0.1–3.0mg/kg) were tested before (acute) and during (chronic) injections (45days) of an individually selected, repeatedly administered dose that reduced the number of food pellets earned by at least 50% compared to when saline was tested acutely. There was a dose-dependent decrease in lever-press rates and food pellets earned. The development of tolerance was quantified by dividing the area-under-the-curve (AUC) of the chronic dose–effect function by the AUC of the acute dose–effect function. Tolerance developed to a similar extent in components with and without extra food pellets for rats in both groups. These results indicate that offsetting reinforcement loss, at least as studied here, did not differentially affect tolerance development.

Resurgence in Siamese fighting fish, Betta splendens

Resurgence of previously reinforced responding was investigated in male Siamese fighting fish (Betta splendens). Swimming through a ring produced 15-s mirror presentations according to, with different fish, either a fixed-ratio 1 or a variable-interval 60-s schedule of reinforcement. When responding was stable, a differential-reinforcement-of-other-behavior schedule was substituted for the mirror-presentation schedule. Following this, mirror presentations were discontinued (extinction). During this latter phase, there were transient increases in the ring-swim response relative to the frequency of such responding during the differential-reinforcement-of-other behavior schedule. Resurgence was similar for the fish exposed previously to the fixed-ratio or to the variable-interval schedule. These results extend to Siamese fighting fish a well-established behavioral phenomenon previously not observed in this species or with this response topography, and only rarely reported following the removal of a non-consumable reinforcer.

On the joint control of preference by time and reinforcer-ratio variation

Five pigeons were trained in a procedure in which, with a specified probability, food was either available on a fixed-interval schedule on the left key, or on a variable-interval schedule on the right key. In Phase 1, we arranged, with a probability of 0.5, either a left-key fixed-interval schedule or a right-key variable-interval 30s, and varied the value of the fixed-interval schedule from 5s to 50s across 5 conditions. In Phase 2, we arranged either a left-key fixed-interval 20-s schedule or a right-key variable-interval 30-s schedule, and varied the probability of the fixed-interval schedule from 0.05 to 1.0 across 8 conditions. Phase 3 always arranged a fixed-interval schedule on the left key, and its value was varied over the same range as in Phase 1. In Phase 1, overall preference was generally toward the variable-interval schedule, preference following reinforcers was initially toward the variable-interval schedule, and maximum preference for the fixed-interval schedule generally occurred close to the arranged fixed-interval time, becoming relatively constant thereafter. In Phase 2, overall left-key preference followed the probability of the fixed-interval schedule, and maximum fixed-interval choice again occurred close to the fixed-interval time, except when the fixed-interval probability was 0.1 or less. The pattern of choice following reinforcers was similar to that in Phase 1, but the peak fixed-interval choice became more peaked with higher probabilities of the fixed interval. Phase 3 produced typical fixed-interval schedule responding. The results are discussed in terms of reinforcement effects, timing in the context of alternative reinforcers, and generalized matching. These results can be described by a quantitative model in which reinforcer rates obtained at times since the last reinforcer are distributed across time according to a Gaussian distribution with constant coefficient of variation before the fixed-interval schedule time, changing to extended choice controlled by extended reinforcer ratios beyond the fixed-interval time. The same model provides a good description of response rates on single fixed-interval schedules.This article is part of a Special Issue entitled: SQAB 2012.

Environmental rearing effects on impulsivity and reward sensitivity.

Previous research has indicated that rearing in an enriched environment may promote self-control in an impulsive choice task. To further assess the effects of rearing environment on impulsivity, 2 experiments examined locomotor activity, impulsive action, impulsive choice, and different aspects of reward sensitivity and discrimination. In Experiment 1, rats reared in isolated or enriched conditions were tested on an impulsive choice procedure with a smaller-sooner versus a larger-later reward, revealing that the isolated rats valued the smaller-sooner reward more than the enriched rats. A subsequent reward challenge was presented in which the delay to the 2 rewards was the same but the magnitude difference remained. The enriched rats did not choose the larger reward as often as the isolated rats, reflecting poorer reward discrimination. Impulsive action was assessed using a differential-reinforcement-of-low-rate task, which revealed deficits in the enriched rats. In Experiment 2, rats reared in isolated, standard, or enriched conditions were tested on reward contrast and reward magnitude sensitivity procedures. The rats were presented with 2 levers that delivered different magnitudes of food on variable interval 30-s schedules. Across all tests, the enriched and social rats displayed more generalized responding to the small-reward lever, but a similar response to the large-reward lever, compared with the isolated rats. This confirmed the results of Experiment 1, indicating poorer reward discrimination in the enriched condition compared with the isolated condition. The results suggest that enrichment may moderate reward generalization/discrimination processes through alterations in incentive motivational processes.

Reinforced Behavioral Variability in Humans

This study aimed to assess the role of reinforced behavioral variability in the learning of a 6-digit target sequence (211212) with 3 groups of human participants (n = 39). For the first group (Control), only the target sequence was reinforced. For the second group (Any), the target sequence was reinforced, and any sequence other than the target sequence could be reinforced on a variable interval 60-s schedule. For the third group (Variable), the target sequence was reinforced, and any sequence other than the target sequence could be reinforced on a variable interval 60-s schedule, if it met a variability criterion. The Control group produced the target sequence significantly more often than the Variable group by the end of the experimental sessions. These findings contradict previous studies with rats that have shown that reinforcement of behavioral variability facilitates the learning of difficult response sequences but are consistent with results from previous studies with humans. Potential reasons for this disparity are discussed.

Phenylpiperazine derivatives with selectivity for dopamine D3 receptors modulate cocaine self-administration in rats

This study examined cocaine self-administration after pretreatments with three structurally related compounds that bind selectively to dopamine D3 receptors (D3Rs) relative to the D2 receptor subtype (D2Rs) and exhibit varying intrinsic activities in the forskolin-stimulated adenylyl cyclase assay. The compounds are: a) WC10, a D3R weak partial agonist/antagonist with 42-fold D3R:D2R selectivity, b) WC26, a 51-fold selective D3R partial agonist, c) WC44, a 23-fold selective D3R agonist. Rats were stabilized on a multiple variable-interval 60-s (VI60) schedule with alternating components of sucrose (45 mg pellets) or cocaine reinforcement (0.375 mg/kg, IV) and then tested for effects of the WC compounds (0.0, 1.0, 3.0, 5.6, or 10.0 mg/kg, IP). Another cohort was trained to self-administer cocaine (0.75 mg/kg, IV) on a VI60 schedule then tested with various doses of cocaine available (0.0–1.5 mg/kg, IV) following pretreatment with WC10 (5.6 or 10.0 mg/kg) or WC44 (10.0 mg/kg). WC10 and WC26 decreased both cocaine and sucrose reinforcement rates at the 10.0 mg/kg dose, whereas WC44 decreased only cocaine reinforcement rate at this dose. Furthermore, WC26 and WC44 increased response latency for cocaine but not sucrose. In the cocaine dose–response experiment, WC10 and WC44 flattened the dose-effect function of cocaine reinforcement rate. All compounds decreased spontaneous locomotion. WC10 and WC26 also reduced cocaine-induced locomotion. These results support the targeting of D3Rs for treatments for cocaine dependence. WC26 and WC44, in particular, show promise as they increased the latency to respond for cocaine but not sucrose, suggesting selective reduction of the motivation for cocaine.

Effects of the amount of acquisition and contextual generalization on the renewal of instrumental behavior after extinction

Four experiments with rat subjects examined the role of context during the extinction of instrumental (free-operant) behavior. In all experiments, leverpressing was first reinforced on a variable-interval 30-s schedule and then extinguished before being tested in the extinction and renewal contexts. The results identified three important variables affecting the renewal effect after instrumental extinction. First, ABA and ABC forms of renewal were strengthened by increasing the amount of acquisition training. This suggests that the strength of the association learned during acquisition, or the final level of performance, influences the degree of renewal after extinction. The effect of the amount of training was modulated by the second factor, the degrees of generalization from the acquisition and extinction contexts to the test context. The third variable was acquisition training in multiple contexts, which was shown to strengthen ABC renewal. Methodological, theoretical, and practical implications are discussed.

PREFERENCE AS A FUNCTION OF ACTIVE INTERRESPONSE TIMES: A TEST OF THE ACTIVE TIME MODEL

In this article, we describe a test of the active time model for concurrent variable interval (VI) choice. The active time model (ATM) suggests that the time since the most recent response is one of the variables controlling choice in concurrent VI VI schedules of reinforcement. In our experiment, pigeons were trained in a multiple concurrent similar to that employed by Belke (1992), with VI 20-s and VI 40-s schedules in one component, and VI 40-s and VI 80-s schedules in the other component. However, rather than use a free-operant design, we used a discrete-trial procedure that restricted interresponse times to a range of 0.5-9.0 s. After 45 sessions of training, unreinforced probe periods were mixed with reinforced training periods. These probes paired the two stimuli associated with the VI 40-s schedules. Further, the probes were defined such that during their occurrence, interresponse times were either "short" (0.5-3.0 s) or "long" (7.5-9.0 s). All pigeons showed a preference for the stimulus associated with the relatively rich VI 40-s schedule--a result mirroring that of Belke. We also observed, though, that this preference was more extreme during long probes than during short probes--a result predicted by ATM.

Renewal after the extinction of free operant behavior

Four experiments were performed to explore the role of context in operant extinction. In all experiments, leverpressing in rats was first reinforced with food pellets on a variable interval 30-s schedule, then extinguished, and finally tested in the same and a different physical context. The experiments demonstrated a clear ABA renewal effect, a recovery of extinguished responding when conditioning, extinction, and testing occurred in contexts A, B, and A, respectively. They also demonstrated ABC renewal (where conditioning extinction and testing occurred in contexts A, B, and C) and, for the first time in operant conditioning, AAB renewal (where conditioning, extinction, and testing occurred in contexts A, A, and B). The latter two phenomena indicate that tests outside the extinction context are sufficient to cause a recovery of extinguished operant behavior and, thus, that operant extinction, like Pavlovian extinction, is relatively specific to the context in which it is learned. AAB renewal was not weakened by tripling the amount of extinction training. ABA renewal was stronger than AAB, but not merely because of context A's direct association with the reinforcer.

Transfer of the discriminative stimulus effects of Δ9-THC and nicotine from one operant response to another in rats

Transfer of the discriminative stimulus effects of two drugs from one operant (original-response) to a topographically different response (transfer-response) that was spared drug discrimination training was investigated. Eight rats were trained in a counterbalanced one manipulandum (lever press and nose poke) drug discrimination procedure. Counterbalanced IP administered nicotine (0.3 mg/kg) or Δ(9)-tetrahydrocannabinol (3.0 mg/kg) functioned as discriminative stimuli. S(D) drugs occasioned sessions of food-reinforcement (variable-interval 30-s schedule); S(Δ) drugs occasioned non-reinforcement. The original-response (lever-pressing or nose-poking) was initially reinforced during 30-min S(D) drug sessions, and non-reinforced on the other alternating S(Δ)-drug sessions. Two separate 5-min non-reinforcement tests, counterbalanced by drug order, revealed stimulus control over the original-response by both drugs, which transferred to the transfer-response. Subsequent extinction training of the transfer-response attenuated the original-response response rates with the S(D) drug conditions but had little impact on discriminative control. Discriminative control was reversed for the transfer-response but had little impact on the original-response but, again, reduced response rate. These data demonstrate that (a) discriminative control by two distinct drug states can transfer and modulate a topographically different free-operant response and, (b) as is true for exteroceptive stimuli, drug states that function as antecedents embedded within the operant three-term contingency have differing relationships with the response and the primary reinforcer.

RESISTANCE TO CHANGE AND FREQUENCY OF RESPONSE‐DEPENDENT STIMULI UNCORRELATED WITH REINFORCEMENT

Stimuli uncorrelated with reinforcement have been shown to enhance response rates and resistance to disruption; however, the effects of different rates of stimulus presentations have not been assessed. In two experiments, we assessed the effects of adding different rates of response-dependent brief stimuli uncorrelated with primary reinforcement on relative response rates and resistance to change. In both experiments, pigeons responded on variable-interval 60-s schedules of food reinforcement in two components of a multiple schedule, and brief response-dependent keylight-color changes were added to one or both components. Although relative response rates were not systematically affected in either experiment, relative resistance to presession feeding and extinction were. In Experiment 1, adding stimuli on a variable-interval schedule to one component of a multiple schedule either at a low rate (1 per min) for one group or at a high rate (4 per min) for another group similarly increased resistance to disruption in the components with added stimuli. When high and low rates of stimuli were presented across components (i.e., within subjects) in Experiment 2, however, relative resistance to disruption was greater in the component presenting stimuli at a lower rate. These results suggest that stimuli uncorrelated with food reinforcement do not strengthen responding in the same way as primary reinforcers.

Effects of d-amphetamine on behavior maintained by signaled and unsignaled delays to reinforcement

Four pigeons responded under a progressive-delay procedure. In a signaled-delay condition, a chained variable interval (VI) 30-s progressive time (PT) 4-s schedule was arranged; in an unsignaled-delay condition, a tandem VI 30-s PT 4-s schedule was arranged. Two pigeons experienced a signaled-unsignaled-signaled sequence; whereas, two pigeons experienced an unsignaled-signaled-unsignaled sequence. Effects of saline and d-amphetamine were determined under each condition. At intermediate doses (1.0 and 1.78m/kg) delay functions were shallower, area under the curve was increased, and, when possible, break points were increased compared to saline; these effects were not systematically related to signaling conditions. These effects on control by delay often were accompanied by decreased response rates at 0s. These results suggest that stimulus conditions associated with the delay may not play a crucial role in effects of d-amphetamine and other stimulants on behavior controlled by reinforcement delay.

Rapid acquisition of preference in concurrent schedules: Effects of d-amphetamine on sensitivity to reinforcement amount

In the present study, effects of d-amphetamine on sensitivity to reinforcement amount under concurrent schedules were examined using a rapid-acquisition procedure. Four pigeons key pecked under single concurrent variable-interval 30-s schedules of grain presentation. Two different reinforcer-amount ratios (7:1 and 1:7) changed across sessions according to a 31-step pseudo-random binary sequence (PRBS). After at least four times through the PRBS, response ratios generally tracked the session-to-session changes in amount ratios; estimates of sensitivity ranged from 0.26 to 0.31 across the four pigeons. Effects of a range of doses of d-amphetamine (0.3-5.6mg/kg) then were determined. For 3 of 4 pigeons, at least one dose, which did not dramatically alter overall response output or bias, decreased sensitivity to reinforcement amount. These results suggest that reducing sensitivity of responding to reinforcement amount may be one behavioral mechanism of stimulants, which may have implications for interpreting drug effects on self-control.

Reinforcer satiation and resistance to change of responding maintained by qualitatively different reinforcers.

In previous research on resistance to change, differential disruption of operant behavior by satiation has been used to assess the relative strength of responding maintained by different rates or magnitudes of the same reinforcer in different stimulus contexts. The present experiment examined resistance to disruption by satiation of one reinforcer type when qualitatively different reinforcers were arranged in different contexts. Rats earned either food pellets or a 15% sucrose solution on variable-interval 60-s schedules of reinforcement in the two components of a multiple schedule. Resistance to satiation was assessed by providing free access either to food pellets or the sucrose solution prior to or during sessions. Responding systematically decreased more relative to baseline in the component associated with the satiated reinforcer. These findings suggest that when qualitatively different reinforcers maintain responding, relative resistance to change depends upon the relations between reinforcers and disrupter types.