Omission Training Research Articles

Partial reinforcement extinction and omission effects in the elimination and recovery of discriminated operant behavior.

Three experiments explored how training reinforcement schedules and context influence the elimination and recovery of human operant behavior. In Experiment 1, participants learned a discriminated operant response in Context A before the response was eliminated with extinction in Context B. They then received a final test in each context. Groups were trained with a discriminative stimulus that predicted a reinforced response on either every trial (continuous reinforcement [CRF]) or some of the trials (partial reinforcement [PRF]). Extinction was slower following PRF training (a partial reinforcement extinction effect [PREE]) and extinguished responding increased when tested in Context A ("ABA" renewal). Experiment 2 further confirmed the PREE was obtained equally whether extinction occurred in the training context (Context A) or a new context (Context B) which is consistent with trial-based accounts of the PREE. Experiment 3 used the same design as Experiment 1 to evaluate the influence of training reinforcement on response elimination with an omission contingency. Across the omission training phase in Context B, the decrease in responding occurred more slowly in the PRF-trained group in comparison to the CRF-trained group, perhaps the first demonstration of what might be termed a PRF omission effect. Again, ABA renewal was observed in Context A. Training reinforcement schedule therefore had a similar influence on response elimination with extinction and omission. Elimination and recovery of human instrumental behavior, with extinction or omission, are influenced by training reinforcement schedule and context. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

ABA and ABC Renewal during Ongoing Omission Training

ABA and ABC Renewal during Ongoing Omission Training

Does DRO type matter?: Cycle versus resetting contingencies in eliminating responding

Following lever-press training on variable-interval 30-s schedules, rats were exposed to three types of schedules designed to eliminate lever pressing. The first two were variations on what is called a differential-reinforcement-of-other-behavior (DRO, “zero rate”, or [target response] omission schedule) schedule. Under both variations, reinforcers were scheduled to occur in different conditions after either fixed or variable inter-reinforcer intervals (IRIs). Under one variation each lever press reset the time interval (i.e., “resetting DRO”) and under the other a reinforcer delivery scheduled at the end of an IRI was cancelled by the first response during the IRI (i.e., “cycle DRO”). In another condition reinforcers were delivered independently of responding after fixed or variable time periods. Each of the DRO procedures reduced response rates quickly and to near zero across ten sessions. The time schedules also reduced responding, albeit at a slower rate. The results extend the analogy of omission training to freeoperant avoidance to shock-deletion avoidance schedules.

A-learning: A new formulation of associative learning theory.

We present a new mathematical formulation of associative learning focused on non-human animals, which we call A-learning. Building on current animal learning theory and machine learning, A-learning is composed of two learning equations, one for stimulus-response values and one for stimulus values (conditioned reinforcement). A third equation implements decision-making by mapping stimulus-response values to response probabilities. We show that A-learning can reproduce the main features of: instrumental acquisition, including the effects of signaled and unsignaled non-contingent reinforcement; Pavlovian acquisition, including higher-order conditioning, omission training, autoshaping, and differences in form between conditioned and unconditioned responses; acquisition of avoidance responses; acquisition and extinction of instrumental chains and Pavlovian higher-order conditioning; Pavlovian-to-instrumental transfer; Pavlovian and instrumental outcome revaluation effects, including insight into why these effects vary greatly with training procedures and with the proximity of a response to the reinforcer. We discuss the differences between current theory and A-learning, such as its lack of stimulus-stimulus and response-stimulus associations, and compare A-learning with other temporal-difference models from machine learning, such as Q-learning, SARSA, and the actor-critic model. We conclude that A-learning may offer a more convenient view of associative learning than current mathematical models, and point out areas that need further development.

ABA renewal and spontaneous recovery of operant performance formerly eliminated by omission training

Two experiments with humans explored the ABA renewal and spontaneous recovery of operant performance previously eliminated through a differential reinforcement-of-other-behavior (DRO). Participants played a videogame in which they acquired two operant responses in Phase 1. Then, all participants experienced an omission procedure for the elimination of Response 1 (R1), while Response 2 (R2) continued to be reinforced. For Experiment 1, one group received Phase 2 in a different context from the one in Phase 1, whereas the other group remained in the same context. Finally, both responses were tested. For Experiment 1, testing took place in the original context immediately after the last trial of Phase 2. For Experiment 2, one group returned 24 h later for the test phase, while the other group received the test immediately after the last trial of Phase 2. We found that changes in the temporal and physical contexts only affected the responses that received the omission training by showing both renewal (Experiment 1) and spontaneous recovery (Experiment 2) of R1. The present results extend findings previously reported with rats to humans. Moreover, our results support the proposal which states that contexts play a key role in modulating the reappearance of operant responses that had been suppressed by retroactive interference treatments.

Relapse of operant behavior after response elimination with an extinction or an omission contingency.

The present study compared relapse after responding was eliminated by extinction or omission training in rats. In Experiment 1, lever pressing was reinforced with food pellets in Context A and then eliminated with either extinction or omission training in Context B. The response was then tested in Contexts A and B in either the presence or absence of free food pellets delivered on a random time schedule. All rats showed higher responding when tested in Context A than Context B, and there was little evidence that omission training attenuated this ABA renewal effect. Noncontingent pellets increased responding after extinction but not after omission. However, when responding on the last day of response elimination was compared to responding during the test in the response-elimination context, there was some evidence that omission-trained rats showed a small increase in responding even when tested with free pellets. Results of Experiment 2 suggest this increase was not due to differences in the temporal distribution of pellets during elimination and the test, and that the result might be due to mere removal of the omission contingency, but any such effect is small and difficult to detect statistically. The results provide new information about factors generating relapse after omission training.

Evidencia de renovación instrumental ABC previamente eliminada con los procedimientos de extinción y omisión

One experiment used a free operant procedure with rats to evaluate renewal of a response previously eliminated by using omission training or by using an extinction treatment. All rats were trained to press a lever for food in context A. Then, the operant responses were eliminated in a second context. In this phase, half of the subjects received an omission training (i. e., delivery of food for withholding responding), while the other half received an extinction procedure. Finally, a within-subject ABC renewal test was conducted. The results demonstrated that all rats showed higher levels of responding when they were tested outside extinction context, regardless the procedure used during the elimination phase.

Under the influence: Effects of adolescent ethanol exposure and anxiety on motivation for uncertain gambling-like cues in male and female rats

Gambling disorder (GD) frequently co-occurs with alcohol use and anxiety disorders, suggesting possible shared mechanisms. Recent research suggests reward uncertainty may powerfully enhance attraction towards reward cues. Here, we examined the effects of adolescent ethanol exposure, anxiety, and reward uncertainty on cue-triggered motivation. Male and female adolescent rats were given free access to ethanol or control jello for 20days. Following withdrawal, rats underwent autoshaping on a certain (100%-1) or uncertain (50%-1-2-3) reward contingency, followed by single-session conditioned reinforcement and progressive ratio tasks, and 7days of omission training, during which lever pressing resulted in omission of reward. Finally, anxiety levels were quantified on the elevated plus maze. Here, we found that uncertainty narrowed cue attraction by significantly increasing the ratio of sign-tracking to goal-tracking, particularly amongst control jello and high anxiety animals, but not in animals exposed to ethanol during adolescence. In addition, attentional bias towards the lever cue was more persistent under uncertain conditions following omission training. We also found that females consumed more ethanol, and that uncertainty mitigated the anxiolytic effects of ethanol exposure observed in high ethanol intake animals under certainty conditions. Our results further support that reward uncertainty biases attraction towards reward cues, suggesting also that heightened anxiety may enhance vulnerability to the effects of reward uncertainty. Chronic, elevated alcohol consumption may contribute to heightened anxiety levels, while high anxiety may promote the over-attribution of incentive value to reward cues, highlighting possible mechanisms that may drive concurrent anxiety, heavy drinking, and problematic gambling.

Omission Training Results in More Resurgence than Alternative Reinforcement

Resurgence refers to the reemergence of a previously reinforced response following the extinction of a more recently reinforced response. In a published study, resurgence occurred to a lesser extent following differential reinforcement of alternative behavior (DRA) than differential reinforcement of other behavior (DRO) with pigeons, but this effect has not been replicated with humans. We conducted a within-subject comparison of resurgence following DRA and DRO, using a human-operant preparation. Six college students earned points by clicking a mouse button across two-component multiple schedules. During both baseline components, points were delivered for the first click to a black circle after 2 s. Responding on the black circle decreased during the second phase, using a DRA in one component and a DRO in the other component. We tested for resurgence by terminating point deliveries (extinction) during both components in the third phase. For three of the participants, more resurgence occurred in the component previously associated with DRO than the component previously associated with DRA. The other three participants showed more resurgence in the first component experienced during extinction, regardless of whether that component was associated with DRA or DRO. However, resurgence was exacerbated when the first component during extinction was DRO rather than DRA. Although sequence influenced resurgence, DRA may be preferable to DRO as an intervention when resurgence is a concern.

Learning not to respond: Role of the hippocampus in withholding responses during omission training

Autoshaping is a Pavlovian learning paradigm in which rats experience pairings of a CS and a US independently of their behavior. When the CS is a lever inserted into the test cage and the US is food delivered to an adjacent magazine, many rats acquire a lever-pressing response called ‘sign-tracking’ even though that response has no effect on the occurrence of either the CS or the US. Since these lever presses are always followed by the US, it has been suggested that sign-tracking could be due to unintended reinforcement of the response. To eliminate the possibility of such instrumental learning the omission schedule, in which a response to the CS cancels the US, was introduced. Previous research has shown that training rats on autoshaping and switching them to an omission schedule generally reduces but does not eliminate sign-tracking, suggesting that it may be due to both Pavlovian and instrumental learning. In the present study naive rats trained on an omission schedule sign-tracked less than a control group exposed to random, unpaired CS and US presentations, suggesting that they learned to withhold the lever press response because of the negative contingency between that response and the US. In a second experiment rats with dorsal hippocampus lesions sign-tracked more than sham-lesioned rats on omission schedules, suggesting that this case of learning not to respond is hippocampus-based. This conclusion is consistent with many previous findings on the inability of hippocampal rats to withhold or suppress responding, and with studies suggesting that one form of extinction of learned responses in normal rats is due to competition from hippocampus-based learning not to respond.

Using DRO, behavioral momentum, and self-regulation to reduce scripting by an adolescent with autism.

Review of the Literature Self -Stimulatory Behavior Considerable impairments in both social and communicative behavior affect most individuals with autism (Rutter & Schopler, 1978). For example, research indicates that during vocal exchanges and in social settings, these individuals often produce forms of verbal behavior that significantly differ from the speech produced by individuals without autism (Fine, Bartolucci, & Szatmari, 1994). This verbal behavior is often categorized as self-stimulatory or stereotyped behavior, and it consists of recurring physical movements or vocalizations that serve no obvious function in the external environment (Harris & Wolchik, 1979). When high rates of delayed echolalia (Fine et al.), also known as scripting, occur in conjunction with low rates of appropriate conversational skills, the social relational opportunities for persons with autism may be greatly reduced (Ross, 2002). Additionally, self-stimulation may interfere with learning or performance (Koegel and Covert, 1972). Since self-stimulation can be potentially harmful to the student as a learner and to the individual as a peer, a number of studies have investigated procedures that could decrease this behavior. Laws, Brown, Epstein, and Hocking (1971) decreased self-stimulatory behavior by directing a teacher to remove attention when his students self-stimulated and to reinforce suitable behavior when they attended. Azrin, Kaplan, and Foxx (1973) reduced self-stimulation in nine individuals with mental retardation by instructing them on the suitable use of vocational and recreational materials. The present study adds to this literature by demonstrating a decrease in the production of self-stimulatory behavior of vocal scripting by an adolescent male with autism, obsessive compulsive disorder, and attention deficit hyperactivity disorder through the use of differential reinforcement, self-regulation and behavioral momentum. Differential Reinforcement of Other Behavior (DRO) All forms of differential reinforcement entail reinforcing one response and withholding reinforcement for another response class (Cooper, Heron, Heward, 2007, p. 470). One of the most frequently used types of differential reinforcement is the differential reinforcement of other behavior (DRO) (Wolery, Bailey, & Sugai, 1988). DRO schedules reverse a contingency by delivering reinforcers based on the absence of a target behavior (Baer, Peterson, & Sherman, 1967). This type of reinforcement delivery is sometimes referred to as omission training since delivery of the reinforcer is contingent upon the omission of the target behavior (Weiher & Harman, 1975). DRO is a useful behavior-reduction procedure for several reasons. First, it highlights the use of positive reinforcement while avoiding the use of aversive stimuli. As a result, many adverse side effects may be avoided. Secondly, target behaviors tend to reduce rather quickly under conditions of DRO, especially if a specific replacement behavior is reinforced (Wolery, Bailey & Sugai, 1998). Lastly, DRO has been shown to be useful in decreasing or eliminating a variety of behavioral excesses, physical aggression and tantrums (Allen, Gottselig, & Boylan, 1982). Behavioral Momentum Generally speaking behavioral momentum involves the use of a series of high-probability requests to increase compliance with lower-probability, instructor-issued requests. Over time, in order to increase generalization and maintenance, fading procedures are implemented such that the interval of time between high probability requests to low probability requests is increased and the ratio of high probability requests to low probability requests is decreased (Ray, Skinner & Watson, 1999). Previous research has demonstrated the benefits of behavioral momentum for increasing compliance with low-probability requests across commands, demands, or requests (Ray, Skinner & Watson, 1999). …

Negative automaintenance omission training is effective.

Twelve pigeons were exposed to negative automaintenance contingencies for 17-27 sessions immediately after brief (14-16 sessions) or extended (168-237 sessions) exposure to positive automaintenance contingencies, or after 4-10 sessions of instrumental training. In all conditions, negative automaintenance contingencies virtually eliminated responding, reducing response rates to an average 1.3 responses per min. This reduction in response rate was validated by a model of transition between early and late response rates that assumed exponential transition of rates from one set of contingencies to the next. The model faithfully reproduced cumulative records, and yielded estimates of terminal rates under negative automaintenance that were close to operant level.

Inactivation of dorsolateral striatum enhances sensitivity to changes in the action–outcome contingency in instrumental conditioning

Actions become compulsive when they are no longer controlled by their consequences. Compulsivity can be assessed using the omission procedure in which animals are required to withhold a previously reinforced action to earn reward. The current study tested the hypothesis that inactivation of the dorsolateral striatum (DLS), a structure implicated in habitual behavior, can enhance sensitivity to changes in the action–outcome contingency during omission training, thus leading to a reduction in compulsive responding. Over 10 days rats were trained to press a freely available lever for sucrose reward delivered on interval schedules of reinforcement. After learning to press the lever at a stable and high rate, rats in the omission group received a session in which the rewards were now delayed by pressing the lever; i.e. withholding lever pressing resulted in increased access to reward. A control group was yoked to the omission group and received the same number and pattern of reward delivery but without the omission contingency. Half the rats in each group received infusions of vehicle into the DLS prior to this training whereas the remainder received an infusion of the GABA-A receptor agonist muscimol. On the next day, the effect of these treatments was assessed on a probe test in which the tendency of the various groups to press the lever was assessed in extinction and without drug infusion. Rats that received vehicle infusions prior to the omission session showed complete insensitivity to the newly imposed omission contingency. In contrast, rats given the infusion of muscimol selectively reduced lever pressing compared to yoked controls. Thus, extended training with interval schedules resulted in compulsive lever pressing that prevented the learning of the omission contingency, whereas inactivation of the DLS appeared to enhance the rats’ sensitivity to this change in the action–outcome contingency.

Renewal of eliminated operant performance by context change : A test of ABA and AAB renewal after omission training

Renewal of eliminated operant performance by context change : A test of ABA and AAB renewal after omission training

Renewal of operant performance formerly eliminated by omission or noncontingency training upon return to the acquisition context

Renewal of operant performance formerly eliminated by omission or noncontingency training was explored in two experiments with rats. When pressing a lever was trained with food reinforcement in one context (A) and then eliminated in a second context (B), responding was renewed by returning the rats to the original context (A). This ABA renewal effect was demonstrated in Experiment 1 when the elimination training was an omission procedure (delivery of food for withholding responding) and in Experiment 2 when it was a noncontingency procedure (delivery of food irrespective of responding). Because omission training (differential reinforcement of other behavior) and noncontingency training have been used in applied settings as effective procedures to reduce undesired human behaviors, the clinical implications of our findings for the relapse of undesirable behavior were discussed.

A comparison of the effects of dorsal or median raphe injections of 8-OH-DPAT in three operant tasks measuring response inhibition

Experiments were conducted to examine the effects of reducing the activity of 5-hydroxytryptamine (5-HT) containing neurons originating in the dorsal raphe (DR) or median raphe (MR) on behavioural inhibition resulting from non-reward. Groups of rats equipped with guide cannulae aimed at either the DR or MR were trained to bar press for food reward in three different operant paradigms. The 5-HT agonist 8-OH-DPAT was then infused into either the DR or MR to suppress the activity of 5-HT neurons and the effects on response inhibition resulting from the omission of reward were measured. At doses of 0.2 and 1 μg 8-OH-DPAT injected into the MR increased responding during extinction of a continuously reinforced response. Similar injections into the DR failed to alter responding. In an omission training paradigm, requiring animals to withhold responding for a period of at least 20 s to receive free reward, rats treated with 1 μg 8-OH-DPAT in the MR showed significantly higher levels of responding. Normal inhibition was observed following DR injections of 8-OH-DPAT. In a third paradigm both DR and MR injections of 8-OH-DPAT impaired accuracy of responding in a signalled go/nogo successive discrimination. In the case of DR treatment this resulted from a decrease in responding during reward periods signalled by S+. However, 5 μg 8-OH-DPAT in the MR significantly increased responding during periods of non-reward, signalled by S −. The pattern of results across these paradigms shows that reducing 5-HT activity at the level of the MR results in a failure to demonstrate normal behavioural inhibition induced by non-reward. These results support the notion that MR 5-HT neurons may be involved in controlling behavioural inhibition by detecting signals of non-reward, which then act to suppress ongoing behaviour. Serotonergic neurons arising from the DR to not appear to be involved in mediating behavioural inhibition resulting from omission of reward.

The roles of event contingencies and reinforcement in human autoshaping and omission responding

Four experiments were conducted to test those features deemed to play an important role in the acquisition and/or maintenance of autoshaping and omission responding with human subjects, viz., event contingencies, reinforcement, stimulus generalization, the type of CS employed, conditioned reinforcement, and response topography. One important aspect of these studies is that the boundary conditions were made to resemble those pertaining to the majority of nonhuman animal studies, i.e., magazine training followed by paired CS-US presentations. The results demonstrated the importance of CS-UR or, alternatively, CS-US contiguity together with generalization and a response-reinforcer contingency for autoshaped response initiation and maintenance, respectively. The conditioned reinforcing effect of CS offset appears to play a part in generating and maintaining some responding during omission training, especially during a response-terminated CS procedure. Similar response topography was observed when the form of the UR was compared to that of the CR or autoshaped response. Taken as a whole the results of these experiments with humans are directly comparable to results previously reported using a wide variety of nonhuman animals.

Negative Automaintenance: Performance of Pigeons under Selective Omission Training Procedures

Pigeons were exposed to autoshaping procedures in which 6-second key illuminations (trials) were occasionally followed by food. In Experiment 1, fewer keypecks occurred during trials in which pecks prevented food delivery with a probability of .33, .67, or 1 than during those in which food was presented equally often but pecking was without programmed consequences, although the difference was statistically significant only when the probability of food omission was 1. In general, less responding occurred during trials where pecks prevented food delivery with a probability of 1 than during trials where pecks prevented food delivery with a probability of .33 or .67. In Experiment 2, responding of master birds typically decreased as the probability of food omission was increased from .33 to 1. Yoked birds, which received food as often as master birds but were never directly exposed to omission dependencies, responded significantly more than master birds when the probability of food omission for the latter was .17, .50, .67, .83, and 1, but not when it was .33. For master birds, the stimulus associated with the lowest probability of food omission evoked the majority of responses in simultaneous choice trials. Yoked subjects behaved similarly during choice trials, but sometimes evidenced a preference for white key illuminations not shown by master birds. These findings suggest that negative responsefood dependencies, even when intermittently imposed, can demonstrably affect auto shaped keypecking.

Multiple unit response in reward areas during operant conditioning reinforced by lateral hypothalamic stimulation in the rat

Neural activities of various brain structures were examined during the acquisition and omission of a behavior. Forty-one rats were used. The positive reinforcer was the stimulation of the lateral hypothalamus, the effectiveness of which had been controlled in advance using the self-stimulation procedure. As original training, head movement during the presentation of an auditory signal was rewarded with the stimulation of the lateral hypothalamus. Following the acquisition of the behavior, omission training was given, during which inhibition of head movement was rewarded. In each session, multiple unit activity for 1 sec prior to the onset of the tone was compared with that during 1 sec tone presentation. In both training sessions, increases in unit activity to the tone, associated with the acquisition of the correct response, were found in the dorsomedial thalamus and the ventral tegmentum. In contrast, the neurons in the prefrontal cortex showed an increased response to the tone only during the omission training. These results suggest that the dorsomedial thalamus and the ventral tegmentum are related to the acquisition of the stimulus-reinforcer contingency, while the prefrontal cortex is related to behavioral inhibition in the omission learning.

The Relative Effects of Three Response-Elimination Procedures in Severely/Profoundly and Trainable Retarded Individuals

The present experiment was designed to compare the response-eliminating efficiency of omission training, extinction training, and graduated omission training following VR reinforcement training in severely/profoundly retarded and trainable retarded individuals. In both populations, the results indicate that the graduated omission training procedure reduced responding significantly faster and more thoroughly than did either omission training or extinction training, and that the latter two procedures do not differ in effectiveness from one another.