Delay Of Reinforcement Gradient Research Articles

Pre-asymptotic response rates as a function of the delay-of-reinforcement gradient summation for Catania’s Operant Reserve: A reply to Berg & McDowell (2011)

Catania’s Operant Reserve (COR; Catania, 2005) is a computational model of operant behaviour. In COR, responding depletes the reserve while reinforcement replenishes the reserve. The replenishment to the reserve depends on the location of responses within the most recent inter-reinforcement interval. The rule that maps replenishment to the responses within an inter-reinforcement interval is given by a delay-of-reinforcement gradient (DOR). Previous research (Berg and McDowell, 2011) found that non-linear DORs produce sigmoidal response rates on single variable-interval schedules while a linear DOR produces hyperbolic response rates. Berg and McDowell took these sigmoidal response rates as evidence against the viability of COR for modelling variable-interval performance. However, Berg and McDowell did not consider the effect of the slope of each DOR. The present conjecture is that the response rates from COR can be made hyperbolic by manipulating the area under the DOR. Our results show that the manipulation of the area under the DOR allows COR to produce hyperbolic response rates regardless of the parametric form of the DOR. Hence, if COR is to be used to model single variable-interval performance, the reinforcement effect over time should be taken into consideration through the manipulation of the area under the DOR.

Adjunctive behaviors are operants

Adjunctive behaviors such as schedule-induced polydipsia are said to be induced by periodic delivery of incentives, but not reinforced by them. That standard treatment assumes that contingency is necessary for conditioning and that delay of reinforcement gradients are very steep. The arguments and evidence for this position are reviewed and rejected. In their place, data are presented that imply different gradients for different classes of responses. Proximity between response and reinforcer, rather than contingency or contiguity, is offered as a key principle of association. These conceptions organize a wide variety of observations and provide the rudiments for a more general theory of conditioning.

EFECTOS DE HISTORIA EN GRADIENTES OBTENIDOS ALARGANDO Y ACORTANDO LA DEMORA DE REFORZAMIENTO

In the absence of shaping or otherwise explicitly training lever pressing, six rats were directly exposed to a tandem random-interval 30-s fixed time t-s schedule. This schedule defined an unsignaled, non-resetting delay of reinforcement procedure in which the delay (the fixed time component of the schedule) was either O, 3, 6, 12, or 24 s long. Three rats were exposed to the delays in ascending order (O to 24 s) and three others were exposed to the delays in descending order. Each delay was in effect for 20 sessions. Lever pressing was acquired by all six rats regardless of whether the O-s or the 24-s delay initially was in effect. Response rates generally were higher when delays were shorter. Rats exposed to the delays in an ascending order responded at higher rates at each delay value than did rats exposed to the delays in a descending order. Such results show how past experience influences the delay of reinforcement gradient and suggest that the concept of an absolute delay of reinforcement gradient is untenable.

Models of trace decay, eligibility for reinforcement, and delay of reinforcement gradients, from exponential to hyperboloid

Behavior such as depression of a lever or perception of a stimulus may be strengthened by consequent behaviorally significant events (BSEs), such as reinforcers. This is the Law of Effect. As time passes since its emission, the ability for the behavior to be reinforced decreases. This is trace decay. It is upon decayed traces that subsequent BSEs operate. If the trace comes from a response, it constitutes primary reinforcement; if from perception of an extended stimulus, it is classical conditioning. This paper develops simple models of these processes. It premises exponentially decaying traces related to the richness of the environment, and conditioned reinforcement as the average of such traces over the extended stimulus, yielding an almost-hyperbolic function of duration. The models account for some data, and reinforce the theories of other analysts by providing a sufficient account of the provenance of these effects. It leads to a linear relation between sooner and later isopreference delays whose slope depends on sensitivity to reinforcement, and intercept on that and the steepness of the delay gradient. Unlike human prospective judgments, all control is vested in either primary or secondary reinforcement processes; therefore the use of the term discounting, appropriate for humans, may be less descriptive of the behavior of nonverbal organisms.

Origins of altered reinforcement effects in ADHD

Attention-deficit/hyperactivity disorder (ADHD), characterized by hyperactivity, impulsiveness and deficient sustained attention, is one of the most common and persistent behavioral disorders of childhood. ADHD is associated with catecholamine dysfunction. The catecholamines are important for response selection and memory formation, and dopamine in particular is important for reinforcement of successful behavior. The convergence of dopaminergic mesolimbic and glutamatergic corticostriatal synapses upon individual neostriatal neurons provides a favorable substrate for a three-factor synaptic modification rule underlying acquisition of associations between stimuli in a particular context, responses, and reinforcers. The change in associative strength as a function of delay between key stimuli or responses, and reinforcement, is known as the delay of reinforcement gradient. The gradient is altered by vicissitudes of attention, intrusions of irrelevant events, lapses of memory, and fluctuations in dopamine function. Theoretical and experimental analyses of these moderating factors will help to determine just how reinforcement processes are altered in ADHD. Such analyses can only help to improve treatment strategies for ADHD.

Delay of reinforcement gradients and attention-deficit/hyperactivity disorder (ADHD): The challenges of moving from causal theories to causal models

notwithstanding the many strengths of the dynamic developmental theory, there remain challenges to be overcome before it can be incorporated into a true causal model of attention-deficit/hyperactivity disorder (adhd). these include the development of reliable measures of reinforcement delay gradients, the validation of shortened reinforcement delay as an endophenotype, and the integration of this pathway with other potential pathways.

Conditioned facilitatory modulation of the response to an aversive stimulus in the crab Chasmagnathus

A mild electrical shock delivered to the walking legs of the crab Chasmagnathus granulatus through a fine layer of sea water, induces a running response that declines after repeated stimulation. Herein, a first series of experiments was aimed at conditioning the response to a light or a dark pulse, but no detectable CR was disclosed during training nor in a later test with the CS alone, even though several stimulus parameters were assayed. In a second series, crabs were repeatedly exposed during training to a light pulse (CS) immediately followed by shock (UCS), and after a 6-h rest interval, tested with either CS-UCS or UCS. The CS-UCS presentation at testing elicited a higher recovery (potentiated recovery, circa 70%) than that evoked by the UCS alone (unconditioned recovery, circa 35%) (Experiment 3). An experiment including the explicity unpaired control procedure (Experiment 4), confirmed that the enhanced response (CR) to the shock (UCS) (measured as potentiated recovery) is conditioned to an illumination signal (CS). An alternative explanation of the potentiated recovery in terms of retardation of habituation proved hardly tenable (Experiment 5). In Experiment 6, crabs were exposed to a variety of light pulse shock and dark pulse shock temporal relationships, and results were found consistent with a delay of reinforcement gradient when light termination was considered as CS. Thus a decrease in luminance is able to become associated with shock and to condition an enhanced response regardless of light termination being the offset of a light pulse or the onset of a dark pulse. Accordingly, potentiated recovery was shown when the shock at test trial was immediately preceded both by a short light pulse and by the end of a long period of illumination (Experiment 7).

Role of short-term processes in the conditioned stimulus preexposure effect and the delay of reinforcement gradient in long-delay taste-aversion learning.

Role of short-term processes in the conditioned stimulus preexposure effect and the delay of reinforcement gradient in long-delay taste-aversion learning.

Role of short-term processes in the conditioned stimulus preexposure effect and the delay of reinforcement gradient in long-delay taste-aversion learning.

Role of short-term processes in the conditioned stimulus preexposure effect and the delay of reinforcement gradient in long-delay taste-aversion learning.

A delay of reinforcement gradient and correlated reinforcement in the instrumental conditioning of conversational behavior.

A delay of reinforcement gradient and correlated reinforcement in the instrumental conditioning of conversational behavior.

Conflict Resolution and the Process of Temporal Integration

Four experiments using rats as Ss are reported. First, delay of reward and delay of punishment gradients were determined empirically for a similar right-left position-discrimination response, using the same apparatus. Second, this empirical information was used to describe how a double approach-avoidance conflict would be resolved when the rewards and punishments for both the right- and left-choice were unevenly distributed in time. The purposes of the experiments were: (a) to demonstrate that the process of resolving such temporal conflicts requires S to integrate over a delay interval the relative value of both the reward and the punishment for each of the two choices, and (b) to show that this value was, in turn, jointly determined by the separate delay of reward and delay of punishment gradients. Further, it was shown that the height and duration of these two types of delay of reinforcement gradients were determined by other variables, and when the height or duration of one of the gradients was altered, so was the underlying value of the reinforcement and the resolution of a temporal conflict. The relation of this process, and of delay of reinforcement in general, to neurotic or self-defeating behavior was noted.

Delay of reinforcement gradients in children’s learning

Ninety Grade 2 children were randomly assigned to one of nine conditions in a study of the effects of concurrent delay of reward and delay of punishment on children’s learning. Three levels of delay of reward and three levels of delay of punishment, 0, 10, and 30 sec. in each case, were selected for study. Both increasing delay of reward and increasing delay of punishment significantly increased the number of errors made during learning.