Models of trace decay, eligibility for reinforcement, and delay of reinforcement gradients, from exponential to hyperboloid

Abstract

Behavior such as depression of a lever or perception of a stimulus may be strengthened by consequent behaviorally significant events (BSEs), such as reinforcers. This is the Law of Effect. As time passes since its emission, the ability for the behavior to be reinforced decreases. This is trace decay. It is upon decayed traces that subsequent BSEs operate. If the trace comes from a response, it constitutes primary reinforcement; if from perception of an extended stimulus, it is classical conditioning. This paper develops simple models of these processes. It premises exponentially decaying traces related to the richness of the environment, and conditioned reinforcement as the average of such traces over the extended stimulus, yielding an almost-hyperbolic function of duration. The models account for some data, and reinforce the theories of other analysts by providing a sufficient account of the provenance of these effects. It leads to a linear relation between sooner and later isopreference delays whose slope depends on sensitivity to reinforcement, and intercept on that and the steepness of the delay gradient. Unlike human prospective judgments, all control is vested in either primary or secondary reinforcement processes; therefore the use of the term discounting, appropriate for humans, may be less descriptive of the behavior of nonverbal organisms.