Free-operant Response Research Articles

Differential effects of clozapine, metoclopramide, haloperidol and risperidone on acquisition and performance of operant responding in rats.

Prior research has not systematically investigated the effects of systemic antipsychotic drugs on operant response acquisition, specifically their behavioural microstructure, reinforcement blunting and relative potency in acquisition compared to performance once operant responding has stabilized. This study aims to systematically investigate the effects of systemically administered clozapine, metoclopramide, haloperidol and risperidone during free operant response acquisition and performance. Following magazine training, food-restricted male Wistar rats lever pressed for food reward in 15 min daily operant conditioning sessions. All drugs suppressed operant response acquisition and performance. Risperidone and metoclopramide, but not clozapine or haloperidol, suppressed operant responding more potently during acquisition than performance. The dopamine D2-like receptor antagonists haloperidol and metoclopramide that affect the ventral and dorsal striatum blunted reinforcement and decreased inactive lever presses in acquisition. In contrast, the atypical antipsychotics clozapine and risperidone that affect the ventral striatum and prefrontal cortex failed to decrease inactive lever presses during acquisition, suggesting a possible decision-making deficit. Haloperidol decreased active lever pressing over performance days. The drugs did not appear to affect rats' sensitivity to active lever press outcome, even though they suppressed active lever pressing. Results suggest that reinforcement impact during operant acquisition is dependent on dopamine D2 receptors while drugs affecting, among other areas, the prefrontal cortex produce a deficit in ability to suppress inactive lever press responses.

The negative effects of alcohol hangover on high-anxiety phenotype rats are influenced by the glutamate receptors of the dorsal midbrain

Alcoholism is a chronic disorder characterized by the appearance of a withdrawal syndrome following the abrupt cessation of alcohol intake that includes symptoms of physical and emotional disturbances, anxiety being the most prevalent symptom. In humans, it was shown that anxiety may increase the probability of relapse. In laboratory animals, however, the use of anxiety to predict alcohol preference has remained difficult. Excitatory amino acids as glutamate have been implicated in alcohol hangover and may be responsible for the seizures and anxiety observed during withdrawal. The dorsal periaqueductal gray (DPAG) is a midbrain region critical for the modulation/expression of anxiety- and fear-related behaviors and the propagation of seizures induced by alcohol withdrawal, the glutamate neurotransmission being one of the most affected. The present study was designed to evaluate whether low- (LA) and high-anxiety rats (HA), tested during the alcohol hangover phase, in which anxiety is the most prevalent symptom, are more sensitive to the reinforcing effects of alcohol when tested in a voluntary alcohol drinking procedure. Additionally, we were interested in investigating the main effects of reducing the excitatory tonus of the dorsal midbrain, after the blockade of the ionotropic glutamate receptors into the DPAG, on the voluntary alcohol intake of HA and LA motivated rats that were made previously experienced with the free operant response of alcohol drinking. For this purpose, we used local infusions of the N-metil D-Aspartato (NMDA) and α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA)-kainate receptors antagonist DL-2-Amino-7-phosphonoheptanoic acid – DL-AP7 (10nmol/0.2μl) and l-glutamic acid diethyl ester – GDEE (160nmol/0.2μl), respectively. Alcohol intoxication was produced by 10 daily bolus intraperitonial (IP) injections of alcohol (2.0g/kg). Peak-blood alcohol levels were determined by gas-chromatography analysis in order to assess blood-alcohol content. Unconditioned and conditioned anxiety-like behavior was assessed by the use of the fear-potentiated startle procedure (FPS). Data collected showed that anxiety and alcohol drinking in HA animals are positively correlated in animals that were made previously familiarized with the anxiolytic effects of alcohol. In addition, anxiety-like behavior induced during alcohol hangover seems to be an effect of changes in glutamatergic neurotransmission into DPAG possibly involving AMPA/kainate and NMDA receptors, among others.

Towards a mathematical model of within-session operant responding.

Operant response rate changes within the course of a typical free-operant experimental session. These changes are orderly, and reliably demonstrated with subjects from different species, responding under different experimental conditions. Killeen (1995) postulated that the response rate changes are a function of the interplay between arousal and satiation and offered a mathematical model for this hypothesis. Here we analyze Killeen's model, demonstrating that, although solid in its principles, it presents some flaws in its implementation. Then, based on the same principles, we build and test a new model of within-session motivation dynamics. We also demonstrate that, by representing arousal as a variable that ranges between 0 and 1, we can obtain a surprisingly simple model of free-operant response rate.

A reinforcement learning approach to instrumental contingency degradation in rats

Goal-directed action involves a representation of action consequences. Adapting to changes in action-outcome contingency requires the prefrontal region. Indeed, rats with lesions of the medial prefrontal cortex do not adapt their free operant response when food delivery becomes unrelated to lever-pressing. The present study explores the bases of this deficit through a combined behavioural and computational approach. We show that lesioned rats retain some behavioural flexibility and stop pressing if this action prevents food delivery. We attempt to model this phenomenon in a reinforcement learning framework. The model assumes that distinct action values are learned in an incremental manner in distinct states. The model represents states as n-uplets of events, emphasizing sequences rather than the continuous passage of time. Probabilities of lever-pressing and visits to the food magazine observed in the behavioural experiments are first analyzed as a function of these states, to identify sequences of events that influence action choice. Observed action probabilities appear to be essentially function of the last event that occurred, with reward delivery and waiting significantly facilitating magazine visits and lever-pressing respectively. Behavioural sequences of normal and lesioned rats are then fed into the model, action values are updated at each event transition according to the SARSA algorithm, and predicted action probabilities are derived through a softmax policy. The model captures the time course of learning, as well as the differential adaptation of normal and prefrontal lesioned rats to contingency degradation with the same parameters for both groups. The results suggest that simple temporal difference algorithms with low learning rates can largely account for instrumental learning and performance. Prefrontal lesioned rats appear to mainly differ from control rats in their low rates of visits to the magazine after a lever press, and their inability to initially detect weak contingency changes.

Transfer of the discriminative stimulus effects of Δ9-THC and nicotine from one operant response to another in rats

Transfer of the discriminative stimulus effects of two drugs from one operant (original-response) to a topographically different response (transfer-response) that was spared drug discrimination training was investigated. Eight rats were trained in a counterbalanced one manipulandum (lever press and nose poke) drug discrimination procedure. Counterbalanced IP administered nicotine (0.3 mg/kg) or Δ(9)-tetrahydrocannabinol (3.0 mg/kg) functioned as discriminative stimuli. S(D) drugs occasioned sessions of food-reinforcement (variable-interval 30-s schedule); S(Δ) drugs occasioned non-reinforcement. The original-response (lever-pressing or nose-poking) was initially reinforced during 30-min S(D) drug sessions, and non-reinforced on the other alternating S(Δ)-drug sessions. Two separate 5-min non-reinforcement tests, counterbalanced by drug order, revealed stimulus control over the original-response by both drugs, which transferred to the transfer-response. Subsequent extinction training of the transfer-response attenuated the original-response response rates with the S(D) drug conditions but had little impact on discriminative control. Discriminative control was reversed for the transfer-response but had little impact on the original-response but, again, reduced response rate. These data demonstrate that (a) discriminative control by two distinct drug states can transfer and modulate a topographically different free-operant response and, (b) as is true for exteroceptive stimuli, drug states that function as antecedents embedded within the operant three-term contingency have differing relationships with the response and the primary reinforcer.

489. Laboratory measures of impulsivity in hospitalized adolescents with disruptive behavior disorders

Disruptive behavior disorders (DBD) in DSM-IV include Conduct Disorder (CD) and Oppositional Defiant Disorder (ODD), both of which are characterized by impulsiveness, such as acting without thinking. We attempted to assess the external validity of a variety of computerized measures designed to prospectively measure “impulsiveness” as well as better characterize impulsiveness in DBD. We tested laboratory impulsiveness in adolescent psychiatric inpatients with clinically significant impulse control problems, all of whom met DSM-IV criteria for either CD or ODD. We compared their laboratory impulsiveness with that of adolescent controls. Compared to controls, adolescents with DBD emitted significantly increased rates of “false alarm” responses (to similar but nontarget stimuli) on a difficult variant of the Continuous Performance Test (CPT) with little difference in rates of “hits”. In addition, compared to the controls, patients with DBD displayed a significant inability to inhibit responses to a target “go” stimulus when presented with a “no-go” stimulus 150–350 milliseconds after “go” stimulus presentation. Moreover, in a free operant task which featured conditions of increasing reward (money) with the passage of time since the last response, the adolescents with DBD showed a relative inability to delay free-operant responses for reward. Differences between adolescents with DBD and controls in serotonin-related alleles will be examined in relation to diagnosis and performance on laboratory measures of impulsivity.

The effects of reinforcer choice on rates of challenging behavior and free operant responding in individuals with severe disabilities

A multielement design was used to evaluate the eAects of three reinforcement conditions on the free-operant and challenging responses of two individuals with severe disabilities. Preference assessments identified three stimuli that could function as reinforcers for each participant. Reinforcers were delivered on a variable ratio schedule for a free-operant response. In the participant-selected condition, each reinforcer delivery consisted of placing an array of three diAerent stimuli in front of the participant, who was allowed to select one. In the constant condition, each reinforcer delivery consisted of placing an array of three identical stimuli in front of the participant. Within each session the three stimuli were held constant, but varied across sessions. In the experimenter-selected condition, each reinforcer delivery consisted of an array of three identical stimuli being placed in front of the participant. Within each session, the three stimuli were presented in an order predetermined by the experimenter such that, on average, each stimulus was available every third reinforcer delivery. For both individuals, the participantselected condition resulted in lower average rates of challenging responses and slightly higher average free-operant response rates. No diAerences in responding were noted between the constant condition and the experimenter-selected condition. #1998 John Wiley & Sons, Ltd. Individuals with severe disabilities frequently require dense reinforcement for both skill acquisition and the reduction of challenging behaviors. Finding and maintaining eAective reinforcers are essential for eAective intervention. Most reinforcement procedures used in applied settings are derived from basic animal research in which establishing operations, such as deprivation, were manipulated (Michael, 1982). Although there is some evidence that manipulating establishing operations can aAect responding in individuals with developmental disabilities

Self-Reported Impulsivity is Correlated with Laboratory-Measured Escape Behavior

Aggression has been previously correlated with impulsive personality. In the present study, Barratt Impulsiveness Scale (BIS) scores of 40 male controls aged 15–40 years were related to the frequency of free-operant aggressive and escape responses toward a fictitious antagonist. Participants earned “points” worth money with repeated button presses on a fixed-ratio schedule and were provoked by the periodic subtraction of a point. These subtractions were blamed on the behavior of a (fictitious) other participant, and aggressive responses (presses of a separate button) were defined as those emitted by the participant with an intent to subtract earnings from the other (fictitious) participant. BIS scores were not correlated with frequency of point-subtracting (aggressive) responses to the point subtractions, but they were correlated with the frequency of escape responses on a third button, which the participant was told would protect his points from subtraction for an unspecified period of time. These results suggest that among normal controls, impulsivity might be characterized by some sensitivity to aversive stimuli.

Visual Reinforcement Signals Interfere with the Effects of Reinforcer Magnitude Manipulations

Three experiments examined the effect of reinforcement magnitude on free-operant response rates. In Experiment 1, rats that received four food pellets responded faster than rats that received one pellet on a variable ratio 30 schedule. However, when the food hopper was illuminated during reinforcer delivery, there was no difference between the rates of response produced by the two magnitudes of reward. In Experiment 2, there was no difference in response rates emitted by rats receiving either one or four pellets of food as reward on a random interval (RI) 60-s schedule. In Experiment 3, rats responding on an RI 30-s schedule did so at a lower rate with four pellets as reinforcement than with one pellet. This effect was abolished by the illumination of the food hopper during reinforcement delivery. These results indicate that the influence of magnitude is obscured by manipulations which signal the delivery of reinforcement.

THE EFFECTS OF RESPONSE COST AND RESPONSE RESTRICTION ON A MULTIPLE‐RESPONSE REPERTOIRE WITH HUMANS

In two experiments a multiple-response repertoire of four free-operant responses was developed with university students as subjects using monetary gain as reinforcement. Following baseline, one of the responses was reduced either by making monetary loss contingent upon it (response cost) or by removing it from the repertoire (response restriction). In Experiment 1 a multielement baseline design was employed in which baseline and restriction or response-cost contingencies alternated semirandomly every 3 minutes. In Experiment 2 a reversal design was employed (i.e., baseline, restriction or response cost, then baseline), and each response required a different amount of effort. Both experiments had the following results: (a) The target response decreased substantially; (b) most nontarget responses increased, and the rest remained near their baseline levels; and (c) no support was found for Dunham's hierarchical, most frequent follower, or greatest temporal similarity rules. For several subjects, the least probable responses during baseline increased most, and the most probable responses increased least. Furthermore, in Experiment 2, responses with the lowest frequency of reinforcement increased most (for all 7 subjects), and those with the greatest frequency of reinforcement increased least (for 5 subjects).

The effects of punishment on unpunished reinforced free-operant responses

Abstract The present experiment assessed the effects of response cost on several unpunished reinforced responses. and thus provided a systematic replication of Crosbie (1990) and a further analysis of Dunham's propositions (Dunham. 1971, 1978; Dunham & Grantmyre, 1982). Four university students made key-press responses on a computer keyboard and won and lost money as reinforcement and punishment, respectively. Ten responses were reinforced, but only one response was punished. Baseline and response cost conditions alternated in a multi-element baseline design. and response cost produced rapid sustained reduction of the target response for 3 of the 4 subjects. Interrupted time-series analysis and the binomial test of multi-element components found that unpunished responses increased for 3 subjects and decreased for 1 subject In contrast to Crosbie (1990). the present nontarget responses did not decrease dramatically when response cost was introduced; they either increased or decreased slightly, or maintaine...

Some Effects of Response Cost on a Free-Operant Multiple-Response Repertoire with Humans

Two experiments with human subjects were performed to investigate the effects of response cost on several unpunished free-operant responses. Using monetary gain on variable-interval schedules as reinforcement, a multiple-response repertoire of button-press responses was developed, then response cost was made contingent on one of these responses. A multielement baseline design was employed in both experiments: During the baseline phase both multielement components were correlated with baseline conditions, then one component was correlated with response cost while baseline conditions continued in the other component. In Experiment 1, a five-response repertoire was developed with no changeover delay (COD), and response cost decreased the target response, increased most nontarget responses, and, for two subjects, one nontarget response decreased. In Experiment 2, a four-response repertoire was developed with a 5-s COD, and similar results were obtained except that no nontarget responses decreased. The present results support my previous findings that punishment affects the whole set of unpunished responses (Crosbie, 1988, 1989, 1990), No support was found, however, for Dunham’s hierarchical rule and most frequent follower propositions (Dunham, 1978; Dunham & Grantmyre, 1982).

The Effects of Punishment on Unpunished Behaviour

The present study examined the effects of response-cost on unpunished free-operant responses. Three university students made key-press responses on a computer keyboard and won and lost money as reinforcement and punishment, respectively. Only one response was ever reinforced or punished, but all responses were recorded. When response-cost was introduced for the target response, both target and non-target responses decreased. Whereas the target response remained at a low level throughout the punishment phase, most non-target responses eventually increased to their baseline levels. Of the three theoretical predictions assessed, support was found only for the proposition that the response that follows the punished response most frequently during baseline is reduced during the punishment phase.

The long term, proactive effect of preshock and ECS as determinants of subsequent responses to punishment

Following alleyway food training, Group 1 was given 3 min of preshock plus ECS (25 mA) 10 sec later on each of 5 days. Group 2 was given preshock and no ECS. Group 3 was given ECS only, and Group 4 was untreated. Group 2 evinced slower alleyway running times than Group 1 and both were more suppressed than Groups 3 and 4 on early food retraining trials prior to test shock. Following retraining, test shock in the alley resulted in the slowest running times for Group 2, and an intermediately slow performance for Group 1 relative to the ECS-only and control groups on early shock recovery trials. For Experiment 2 ECS durations of 0.4 and 0.8 sec were combined factorially with ECS administration on each of 5 or 10 days (4 groups of 9 rats each). No-ECS and hip-shock control groups were included. Test shock given 7 days after treatment resulted in marked suppression of a stable free operant response in both 0.4 sec groups relative to the remaining four. Experiment 3 involved a replication of Experiment 1 with the single exception that the ECS duration was increased from 0.4 to 0.8 sec. The results of Experiment 3 basically paralleled those of Experiment 1.

Free operant response reinstatement during extinction and time-contingent (DRO) reward.

3 experiments are reported on the reinstatement of a bar-press response by rats during free operant extinction. After extended training on a VR schedule of food reward, an extinction procedure in which reward was negatively correlated with the bar-press response was initiated, i.e., reward was presented after 4 min of no bar presses. Results indicated that pause-contingent reward served to increase the rate of responding beyond what could be explained by the eliciting properties of reward-correlated stimuli or the “novelty” of stimulus change.

Stimulus properties of reinforcement during extinction of a free operant response

Six moderately retarded adolescent girls were conditioned to respond at high stable rates for 4 consecutive days on a fixed ratio 50 schedule. They were then placed on extinction until they met a variable pause criterion for not responding. They were then delivered free reinforcement. The first time that free reinforcement was delivered there was an immediate resumption of responding by all six Ss. The effect ranged from a transitory effect lasting for only one reinforcement presentation to a rather permanent effect showing no decrease after more than 100 reinforcement presentations.

Cold acclimatization in hairless mice measured by behavioral thermoregulation

For 30 successive days, mice were administered six-hour periods in the cold (48 °F) and permitted to switch on a heat lamp by means of a free operant response. In the course of this training, the magnitude of the fall in skin temperature in the course of a session declined. The frequency of heat rewards also declined, indicating the development of cold acclimatization.

Acquisition of operant differentiation and discrimination in institutionalized retarded children.

A device for simultaneous measurement of free-operant response differentiation and stimulus discrimination demonstrated specific deficits in the two functionally independent processes. A constant apparatus program revealed reliable individual differences in speed, sequence and efficiency of acquisition and in performance stability. Neither age, psychometric scores, verbal facility, nor duration of institutionalization accurately predicted individual performance. Continuum extremes were related to school progress.

Interaction between Effort and Partial Reinforcement in Free Operant Response of Rats and Fish

Rats and fish were trained to give a free operant response under partial versus consistent reinforcement and high versus low effort. Resistance to extinction was then measured in terms of number of responses to criterion (R), time to criterion (T), and rate to criterion (R/T). Effort failed to affect any of these measures in either species. In rats partial reinforcement produced greater resistance to extinction than consistent reinforcement on R and T, but not on R/T, whereas partially reinforced fish failed to differ from those consistently reinforced on R and T bur exceeded the consistently reinforced fish on R/T. The R and T measures were correlated in both groups of fish but only in the consistently reinforced rats. Partially reinforced fish were less variable than consistently reinforced fish on the T measure. It is suggested that the principal effect of partial reinforcement in rats is on mean time to extinction, but the effect in fish is on variability of time to extinction.

The relationship between paired comparisons scale values of stimuli and their function as reinforcers of a free operant response with young children

An experiment to determine whether pre-school children respond to stimuli as reinforcers in an experimental situation consistent with their paired comparisons preferences. Twenty-seven kindergarten Ss individually rated 4 stimuli using paired comparisons procedures. Each S was then reinforced for lever pulling responses with token reinforcers redeemable for the S’s preference among the 4 stimuli. Results indicated low consistence of paired comparison scale values and no relationship of paired comparison and operant preferences. The results suggest caution should be exercised in extrapolating previously reported paired comparisons findings to laboratory learning research with young children.