Reinforcement History Research Articles

Response Systems, Antagonistic Responses, and the Behavioral Repertoire.

While response systems are often mentioned in the behavioral and physiological literature, an explicit discussion of what response systems are is lacking. Here we argue that response systems can be understood as an interaction between anatomically constrained behavioral topographies occasioned by currently present stimuli and a history of reinforcement. “New” response systems can develop during the lifetime as the organism gains instrumental control of new fine-grained topographies. Within this framework, antagonistic responses compete within each response system based on environmental stimulation, and competition is resolved at the striatum-thalamo-cortical loops level. While response systems can be by definition independent from one another, separate systems are often recruited at the same time to engage in complex responses, which themselves may be selected by reinforcement as functional units.

Sunk cost effects hinge on the neural recalibration of reference points in mental accounting.

The context of reinforcement history drastically influences human value-based choices. Mental accounting theory concerns how prior outcomes are perceived, combined and assigned into specific "mental" accounts to influence subsequent decisions but remains agnostic about the underlying computational and neural mechanisms. In a two-stage sequential decision-making task, we found previously incurred costs and bonuses biased subjects' choices in the opposite directions with similar magnitudes. Such effects were consistent with a computational model where the reference point was recalibrated by prior gains and losses encoded in the ventral striatum activities. Moreover, individual's susceptibility to prior outcomes was captured by the response of the dorsolateral prefrontal cortex and its functional connectivity with the medial orbitofrontal cortex, whose activity tracked the value of the chosen option. Our findings provide both behavioral and neural evidence of how sunk costs, benefits, and prospects are integrated within the mental accounting framework to influence choice behavior.

Post-error Slowing During Instrumental Learning is Shaped by Working Memory-based Choice Strategies

Post-error slowing (PES) – a relative increase in response time for a decision on trialtgiven an error on trialt − 1 – is a well-known effect in studies of human decision-making. Post-error processing is reflected in neural signatures such as reduced activity in sensorimotor regions and increased activity in medial prefrontal cortex. PES is thought to reflect the deployment of executive resources to get task performance back on track. This provides a general account of PES that cuts across perceptual decision-making, memory, and learning tasks. With respect to PES and learning, things are complicated by the fact that learning often reflectsmultiple qualitatively different processes with distinct neural correlates. It is unclear if multiple processes shape PES during learning, or if PES reflects a policy for reacting to errors generated by one particular process (e.g., cortico-striatal reinforcement learning). Here we provide behavioral and computational evidence that PES is influenced by the operation of multiple distinct processes. Human subjects learned a simple visuomotor skill (arbitrary visuomotor association learning) under low load conditionsmore amenable to simple working memory-based strategies, and high load conditions that were putatively more reliant on trial-by-trial reinforcement learning. PES decreased withload, even when the progress of learning (i.e., reinforcement history) was accounted for. This result suggested that PES during learning is influenced by the recruitment of working memory. Indeed, observed PES effects were approximated by a computational model with parallel working memory and reinforcement learning systems that are differentially recruited according to cognitive load.

Avoidance begets avoidance: A computational account of negative stereotype persistence.

Research on stereotype formation has proposed a variety of reasons for how inaccurate stereotypes arise, focusing largely on accounts of motivation and cognitive efficiency. Here, we instead consider how stereotypes arise from basic processes of approach and avoidance in social learning. Across five studies, we show that initial negative interactions with some members of a group can cause subsequent avoidance of the entire group, and that this avoidance perpetuates stereotypes in two ways. First, when information gain is contingent on approaching the target, avoidance restricts the information available with which to update one's beliefs. Second, computational models that consider the perceiver's full reinforcement history demonstrate that avoidance directly reinforces itself, such that initial avoidance of group members increases the probability of later acts of avoidance toward that group. Finally, we find initial evidence for a potential dissociation between behavior and explicit beliefs, with avoidance reinforcing avoidant behaviors without necessarily affecting self-reported beliefs. Overall, these results suggest that avoidance behaviors toward members of social groups can perpetuate inaccurate negative beliefs and expectations about those groups, such that initial interactions with a group have a compounding effect on overall impressions. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Examining effects of training duration on humans' resurgence and variability using a novel touchscreen procedure.

Resurgence occurs when a previously reinforced and then extinguished target response increases due to reducing/eliminating an alternative source of reinforcement or punishing an alternative response. We evaluated whether duration of reinforcement history for a target response (1) affects the degree to which resurgence is observed in humans and (2) produces different gradients of response generalization around target responding during extinction testing. We arranged a novel touchscreen interface in which university students could swipe a 3D soccer ball to spin any direction. In Phase 1, the first direction swiped became the target and produced points exchangeable for money for 3 or 1 min across 2 groups. The first swipe was recorded but had no programmed consequence in a third group. In Phase 2, swipes 180-degrees from the target resulted in points for 3 min in all groups. Point deliveries ceased for 2 min to test for resurgence in Phase 3. Target responses resurged during testing to a relatively greater extent with longer Phase-1 training but gradients of response generalization did not differ among groups. These findings extend prior research on the role of training duration on resurgence. We discuss methodological and conceptual issues surrounding the assessment of response generalization in resurgence.

New Caledonian crows' planning behaviour: a reply to de Mahy et al .

New Caledonian crows' planning behaviour: a reply to de Mahy <i>et al</i> .

Responding Fails to Extinguish During Human-Laboratory Experiments of Resurgence

Resurgence is observed when a previously extinguished behavior reemerges while a more recently reinforced behavior is extinguished. Resurgence is further defined as responding that is greater than an inactive control response that has never produced reinforcement. Recent studies of resurgence using neurotypical adults as participants in human-laboratory investigations have produced discrepant patterns of responding compared to nonhuman animal laboratory studies when comparing control response performance. Namely, human-laboratory investigations have produced no differences between target and control responding, and persistence of all response types across the resurgence-test phase. In the present study, we conducted two human-laboratory experiments to determine if these effects were a product of the history of reinforcement associated with the target response as well as the types of technology used in human-laboratory studies. For all participants, we found no differences in levels of resurgence and occurrence for the target and control response, respectively. Moreover, we observed persistence of all response types across the resurgence-test phase in a manner consistent with prior research. This finding was apparent even when the length of baseline (i.e., reinforcement for the target) was increased, when the length of extinction was increased, and when low-technology stimuli were used. We highlight the implications of this outcome in the context of recent human-laboratory studies that have used arbitrary responses to study resurgence, and discuss the possible role of verbal mediation in these investigations.

Atypical Reinforcement Learning in Developmental Dyslexia.

According to the Procedural Deficit Hypothesis, abnormalities in corticostriatal pathways could account for the language-related deficits observed in developmental dyslexia. The same neural network has also been implicated in the ability to learn contingencies based on trial and error (i.e., reinforcement learning [RL]). On this basis, the present study tested the assumption that dyslexic individuals would be impaired in RL compared with neurotypicals in two different tasks. In a probabilistic selection task, participants were required to learn reinforcement contingencies based on probabilistic feedback. In an implicit transitive inference task, participants were also required to base their decisions on reinforcement histories, but feedback was deterministic and stimulus pairs were partially overlapping, such that participants were required to learn hierarchical relations. Across tasks, results revealed that although the ability to learn from positive/negative feedback did not differ between the two groups, the learning of reinforcement contingencies was poorer in the dyslexia group compared with the neurotypicals group. Furthermore, in novel test pairs where previously learned information was presented in new combinations, dyslexic individuals performed similarly to neurotypicals. Taken together, these results suggest that learning of reinforcement contingencies occurs less robustly in individuals with developmental dyslexia. Inferences for the neuro-cognitive mechanisms of developmental dyslexia are discussed.

A Report on the History and Development of Soil Reinforcement

A Report on the History and Development of Soil Reinforcement

Effects of reinforcement and response-cost history on instructional control.

The present study compared the effects of reinforcement or punishment versus no additional consequences for instruction following on instructional control and subsequent rule-governed insensitivity. In two experiments, adult participants were presented with repeated choices between a short progressive-time schedule and either a fixed time-schedule or a longer progressive-time schedule. In Experiment 1, three groups were given an initially accurate instruction relative to the direct contingency. A control group experienced no additional consequences for compliance with instructions, whereas compliance resulted in additional points for a second group, and noncompliance led to the subtraction of points for a third group. In a subsequent phase, instructions became inaccurate and there were no additional consequences for compliance or noncompliance for any group. Consistent with previous results, rule-governed insensitivity was observed in all participants. Experiment 2 employed the same procedure, except instructions were inaccurate throughout all sessions, and compliance in the subsequent phase resulted in diminishing points per session. Reinforcement for following instructions increased instructional control and subsequent rule-governed insensitivity. This increase was maintained even after the termination of additional consequences, a result that supports theoretical suggestions that a history of reinforcement for complying with instructions and rules is an important factor in rule-governed insensitivity.

Eating expectancies and reinforcement learning: a state-based test of affect regulation and expectancy models in the natural environment.

Affect regulation and expectancy-based models suggest that improvement in affect following binge-eating (BE) episodes contributes to increased eating expectancies, which then promote BE maintenance. The current pilot study utilized ecological momentary assessment to examine the prospective independent and interactive effects of eating reinforcement experiences [operationalized as reductions in negative affect (NA) following BE episodes] and eating expectancies on subsequent BE behavior among 17 women with recurrent BE. Greater reductions in momentary NA following a BE episode (i.e., greater reinforcement) predicted higher levels of eating expectancies on the following day. Further, current eating expectancies interacted with proximal reinforcement history to predict future BE episodes. Participants were more likely to report BE episodes on days that were characterized by higher eating expectancies and preceded immediately by a day during which they experienced greater reinforcement from BE. These preliminary results are consistent with affect regulation and expectancy-based models of BE, suggesting a dynamic and potentially modifiable process of reward-based learning associated with BE behavior. Level IV, multiple time series.

Emergência de intraverbais a partir do treino de tatos com Precision Teaching

&#x0D; Indivíduos com TEA (Transtorno do Espectro do Autismo) podem apresentar dificuldades de comunicação, tais como déficits na emissão de intraverbais durante interação social. Ao longo dos anos, pesquisadores vêm testando diversos procedimentos para o ensino desta habilidade (Kodak, Fuchtman e Paden (2012); McMorrow, Foxx, Faw, e Bittle (1987); Cihon, White, Zimmerman, Gesick, Stordahl &amp; Eshleman (2017)). Este trabalho objetivou avaliar se o ensino de tatos tematicamente relacionados a níveis fluentes poderia resultar na emergência de intraverbais. Utilizou-se delineamento de linha de base múltipla entre categorias, de sujeito único, e foram realizadas cinco etapas de ensino: (1) treino de tato, (2) treino de fluência de tato, (3) sonda de persistência (endurance), (4) sonda de estabilidade (stability) e (5) sonda de retenção (retention). Os resultados indicaram que a habilidade intraverbal emergiu a partir do treino fluente de tatos tematicamente relacionados com uso de Ensino Preciso (Precision Teaching), sem necessidade de treino direto. Estudos futuros podem investigar a emergência com indivíduos que não emitem intraverbais para nenhuma categoria, e também comparar o tempo de aquisição entre o treino direto e o treino de fluência de tato com um mesmo sujeito. O presente estudo apresenta limitações quanto ao contexto no qual foi realizado e participante (setting terapêutico e história de reforçamento de seguir instruções), e quanto à forma de apresentação dos estímulos (sinalização das categorias temporal e visualmente).

Reinforcement history shapes primary visual cortical responses: An SSVEP study

Efficient learning requires allocating limited attentional resources to meaningful stimuli and away from irrelevant stimuli. This prioritization may occur via covert attention, evident in the activity of the visual cortex. We used steady-state visual evoked potentials (SSVEPs) to assess whether associability-driven changes in stimulus processing were evident in visuocortical responses. Participants were trained on a learned-predictiveness protocol, whereby one stimulus on each trial accurately predicted the correct response for that trial, and the other was irrelevant. In a second phase the task was arranged so that all cues were objectively predictive. Participants’ overt attention (eye gaze) was affected by each cue’s reinforcement history, as was their covert attention (SSVEP responses). These biases persisted into Phase 2 when all stimuli were objectively predictive, thereby demonstrating that learned attentional processes are evident in basic sensory processing, and exert an effect on covert attention above and beyond the effects of overt gaze bias.

MDPV self-administration in female rats: influence of reinforcement history.

A subset of male rats that self-administer 3,4-methylenedioxypyrovalerone (MDPV) have unusually high levels of drug intake; however, factor(s) that influence this behavior (e.g., reinforcement history and sex) are unknown. Characterize the reinforcing potency and effectiveness of MDPV in female rats to determine whether (1) a subset of females also develop high levels of MDPV self-administration (i.e., a high-responder phenotype) and (2) the degree to which the high-responder phenotype is influenced by various reinforcement histories (i.e., responding for cocaine or food). Female Sprague Dawley rats initially responded for MDPV (0.032mg/kg/infusion), cocaine (0.32mg/kg/infusion), or food (45-mg grain pellet) under fixed ratio (FR) 1 and FR5 schedules of reinforcement. After 20 sessions, the cocaine- and food-history rats responded for MDPV for 20 additional sessions. Dose-response curves for MDPV were generated under FR5 and progressive ratio (PR) schedules of reinforcement. A subset of rats responding for MDPV developed high levels of MDPV intake. A history of responding for cocaine, but not food, inhibited the development of high levels of MDPV intake. Large individual differences were observed in the level of self-administration when MDPV was available under an FR5, but not PR, schedule of reinforcement. MDPV functions as a powerful reinforcer in female rats, as has been previously reported in male rats. The substantial variability in MDPV self-administration between subjects may be related to individual differences in human drug-taking behavior.

Honeybees solve a multi-comparison ranking task by probability matching.

Honeybees forage on diverse flowers which vary in the amount and type of rewards they offer, and bees are challenged with maximizing the resources they gather for their colony. That bees are effective foragers is clear, but how bees solve this type of complex multi-choice task is unknown. Here, we set bees a five-comparison choice task in which five colours differed in their probability of offering reward and punishment. The colours were ranked such that high ranked colours were more likely to offer reward, and the ranking was unambiguous. Bees' choices in unrewarded tests matched their individual experiences of reward and punishment of each colour, indicating bees solved this test not by comparing or ranking colours but by basing their colour choices on their history of reinforcement for each colour. Computational modelling suggests a structure like the honeybee mushroom body with reinforcement-related plasticity at both input and output can be sufficient for this cognitive strategy. We discuss how probability matching enables effective choices to be made without a need to compare any stimuli directly, and the use and limitations of this simple cognitive strategy for foraging animals.

Transitive inference in cleaner wrasses (Labroides dimidiatus).

Transitive inference (TI) is the ability to infer unknown relationships from previous information. To test TI in non-human animals, transitive responding has been examined in a TI task where non-adjacent pairs were presented after premise pair training. Some mammals, birds and paper wasps can pass TI tasks. Although previous studies showed that some fish are capable of TI in the social context, it remains unclear whether fish can pass TI task. Here, we conducted a TI task in cleaner wrasses (Labroides dimidiatus), which interact with various client fishes and conspecifics. Because they make decisions based on previous direct and indirect interactions in the context of cleaning interactions, we predicted that the ability of TI is beneficial for cleaner fish. Four tested fish were trained with four pairs of visual stimuli in a 5-term series: A-B+, B-C+, C-D+, and D-E+ (plus and minus denote rewards and non-rewards, respectively). After training, a novel pair, BD (BD test), was presented wherein the fish chose D more frequently than B. In contrast, reinforcement history did not predict the choice D. Our results suggest that cleaner fish passed the TI task, similar to mammals and birds. Although the mechanism underlying transitive responding in cleaner fish remains unclear, this work contributes to understanding cognitive abilities in fish.

Human Choice Predicted by Obtained Reinforcers, Not by Reinforcement Predictors.

Macphail (1985) proposed that “intelligence” should not vary across vertebrate species when contextual variables are accounted for. Focusing on research involving choice behavior, the propensity for choosing an option that produces stimuli that predict the presence or absence of reinforcement but that also results in less food over time can be examined. This choice preference has been found multiple times in pigeons (Stagner and Zentall, 2010; Zentall and Stagner, 2011; Laude et al., 2014) and has been likened to gambling behavior demonstrated by humans (Zentall, 2014, 2016). The present experiments used a similarly structured task to examine adult human preferences for reinforcement predictors and compared findings to choice behavior demonstrated by children (Lalli et al., 2000), monkeys (Smith et al., 2017; Smith and Beran, 2020), dogs (Jackson et al., 2020), rats (Chow et al., 2017; Cunningham and Shahan, 2019; Jackson et al., 2020), and pigeons (Roper and Zentall, 1999; Stagner and Zentall, 2010). In Experiment 1, adult human participants showed no preference for reinforcement predictors. Results from Experiment 2 suggest that not only were reinforcement predictors not preferred, but that perhaps reinforcement predictors had no effect at all on choice behavior. Results from Experiments 1 and 2 were further assessed using a generalized matching equation, the findings from which support that adult human choice behavior in the present research was largely determined by reinforcement history. Overall, the present results obtained from human adult participants are different than those found from pigeons in particular, suggesting that further examination of Macphail (1985) hypothesis is warranted.

Efficacy and efficiency of auditory discrimination procedures for children with autism spectrum disorder and typical development: A preliminary investigation

ABSTRACT An auditory discrimination involves differential behavior in the presence of auditory stimuli due to a history of reinforcement. A dearth of research on the instruction of auditory discrimination is in the behavior-analytic literature, so identifying strategies to teach this skill is necessary. This might be especially pertinent for individuals diagnosed with autism spectrum disorder (ASD) with impaired auditory discrimination repertoires. The current study evaluated the efficacy and efficiency of go/no-go, do this/do that, and auditory match-to-sample on the acquisition of novel auditory discrimination tasks with three children with ASD and two children of typical development in a total of ten comparisons. At least one condition was efficacious for all participants. The do this/do that procedure was more likely to be efficacious and efficient compared to go/no-go and auditory match-to-sample.

"Other" behavior and the DRO: The roles of extinction and reinforcement.

Jessel et al. (2015) provided some evidence to suggest that "other" behavior is strengthened in the differential reinforcement of other behavior (DRO). The present study is a systematic replication of the Jessel et al. procedures. The effects of DRO and extinction on target responding, target-other responding (a response with an established history of reinforcement), and nontarget-other responding emitted by children with intellectual and developmental disabilities and children with no known diagnoses were compared. Other behavior increased in at least one DRO condition for each participant, suggesting that other behavior increases when using DRO, at least initially. Under extinction, target responding and target-other responding decreased to low rates for three of the five participants; however, rates of nontarget-other responding were elevated compared to the DRO condition. These results suggest that increased rates of target-other responding and nontarget-other responding during the DRO condition may be a result of extinction-induced variability.

A new method for dynamically estimating long-term seepage failure frequency for high concrete faced rockfill dams

Failure frequency of a dam varies greatly with reservoir water level and time. It is important but difficult to dynamically estimate failure frequency of high concrete faced rockfill dams (CFRDs) merely based on the probabilistic approach due to limited failure cases of high CFRDs. In this study, a new method is developed for dynamically estimating the long-term revolution of seepage failure frequency of high CFRDs by combining an analogical approach with the Bayesian approach. A simple and precise seepage generalized method and a parameter named defect parameter are introduced to provide the basis of comparison between the dam of interest and the accumulated database of high CFRDs. Using the cumulative probability distribution from the database, the correlation between annual seepage failure frequency and defect parameter is established. A high CFRD case in China with an operation history of slab breakage and reinforcement is selected as an example to illustrate the applicability of the new method. It is verified that the variation tendency of safety level of high CFRD is well simulated through the annual seepage failure frequency calculated by the newly proposed method.