Reinforcement History Research Articles

Choice versus no choice: Practical considerations for increasing choices.

Choice involves engaging in a selection response when multiple options are concurrently available. Choices can be incorporated into many components of behavior-analytic treatment such as providing clients with a choice between multiple items, activities, or tasks. We reviewed the main characteristics of 38 behavior-analytic articles that compared choice and no-choice conditions. We coded the experimental arrangements of choice and no-choice conditions and analyzed potential factors affecting preferences for choice and no choice. The findings suggest that the sizing of alternatives from which to choose, the timing of choice opportunities, and the timing of the delivery of the chosen option varied across the studies. Furthermore, preferences for choice shifted with differential reinforcement history and response effort manipulations of choice or no choice. The findings suggest that individual variables should be considered when providing choices, but more research is needed.

Persistent threat avoidance following negative reinforcement is not associated with elevated state anxiety.

Obsessive-Compulsive Disorder (OCD) is a debilitating illness consisting of obsessions and compulsions. OCD severity and treatment response are correlated with avoidant behaviors thought be performed to alleviate obsession-related anxiety. However, little is known about either the role of avoidance in the development of OCD or the interplay between anxiety states and avoidance behaviors. We have developed an instrumental negative reinforcement (i.e. active avoidance) paradigm in which mice must lever-press to avoid upcoming foot shocks. We show that mice (both sexes) can learn this task with high acquisition rates (75%) and that this behavior is largely stable when introducing uncertainty and modifying task structure. Furthermore, mice continue to perform avoidance responses on trials where lever pressing is not reinforced and increase response rates as they are maintained on this paradigm. With this paradigm, we did not find a relationship between negative reinforcement history and anxiety-related behaviors in well-established anxiety assays. Finally, we performed exploratory analyses to identify candidate regions involved in well-trained negative reinforcement using expression of the immediate early gene c-Fos. We detected correlated c-Fos expression in 1) cortico-striatal regions which regulate active avoidance in other paradigms and 2) amygdala circuits known to regulate conditioned defensive behaviors.Significance Statement Studies in patients with OCD suggest that compulsions are performed to avoid perceived threats and modulate anxiety tied to obsessions and/or compulsions. The negative reinforcement of avoidance and alleviated anxiety could therefore be a key driver of compulsive behaviors. However, there are still outstanding questions concerning the relationship between these two behaviors and the neural circuits involved in mediating negative reinforcement. We have developed an operant negative reinforcement paradigm in mice with discrete avoid and escape behaviors that can be learned without prior reward training with high throughput (75% acquisition) with responding that persists during nonreinforced trials. However, no differences were observed between negative reinforcement vs. unshocked and inescapably shocked controls, suggesting that continued negative reinforcement did not impact anxiety.

Token economies: Evidence‐based recommendations for practitioners

AbstractToken economies are among the oldest and most successful teaching programs in applied behavior analysis. Despite a rich history of basic and applied research on token systems, there remains a research‐to‐practice gap. Our aim in this paper is to bridge this gap between research and application, by providing evidence‐based recommendations and practical guidelines for application of token reinforcement methods. The recommendations in Part 1 are for building a token economy from the ground up, in learners without a history of token reinforcement, whereas those in Part 2 are concerned with existing token economies without regard to how they were established. Although token economies have proven generally effective across a range of settings and populations, they could be even more effective when based on the latest research and theory. Thus, apart from specific recommendations, we hope to show the benefits of an integrated evidence‐based approach to the application of token reinforcement principles in educational and clinical settings.

Evaluating functions of praise for children diagnosed with autism spectrum disorder.

We assessed whether novel praise statements could be used to (a) maintain and increase responses with existing reinforcement histories and (b) teach a previously untaught response among children diagnosed with autism spectrum disorder across two experiments. During response-stimulus pairing, two responses resulted in preferred edibles but only one also produced a praise statement. In the absence of edibles, the response continuing to produce praise tended to persist more. Next, reversing the praise contingency tended to increase the other response. However, in no case did contingent delivery of those same praise statements result in the acquisition of untaught responses. These findings suggest that conditioning praise statements could serve different functions (antecedent or consequence) depending on the reinforcement history for particular responses.

A novel approach to evaluation of lumbar bone density using Hounsfield units in volume of interest on computed tomography imaging.

The purpose of this retrospective study was to evaluate the relationship between bone mineral density (BMD), as assessed with dual-energy x-ray absorptiometry (DEXA), and Hounsfield units (HU) measured in volumes of interest (VOIs) and regions of interest (ROIs) on lumbar spine CT. A retrospective analysis was performed on data of lumbar vertebrae obtained from patients who underwent both DEXA and lumbar spine CT scan within a 6-month period. Vertebrae with a history of compression fracture, infectious spondylitis, cement reinforcement, or lumbar surgery were excluded. HU measurements were performed in the VOI and ROI (midaxial, midcoronal, and midsagittal sections) with CT, whereas BMD was assessed with DEXA. Statistical analyses, including correlation assessments and receiver operating characteristic (ROC) curve analyses, were performed. This analysis included 712 lumbar vertebrae, with a median patient age of 72.0 years. BMD values and HU measurements in the VOI increased sequentially from L1 to L4, whereas HU values in the ROI did not show a consistent pattern. HU values in the VOI consistently showed a stronger correlation with BMD than those in the ROI. ROC analysis revealed patient-level cutoff values for the diagnosis of osteoporosis at different lumbar vertebral levels with high sensitivity and specificity, as well as an excellent area under the curve. This is the first study to introduce a novel approach using the HU value in the VOI to assess bone health at the lumbar spine. There is a strong correlation between the HU value in the VOI and BMD, and the HU value in the VOI can be used to predict osteoporosis.

Influence of reinforcement and its omission on trial-by-trial changes of response bias in perceptual decision making.

Discrimination performance in perceptual choice tasks is known to reflect both sensory discriminability and nonsensory response bias. In the framework of signal detection theory, these aspects of discrimination performance are quantified through separate measures, sensitivity (d') for sensory discriminability and decision criterion (c) for response bias. However, it is unknown how response bias (i.e., criterion) changes at the single-trial level as a consequence of reinforcement history. We subjected rats to a two-stimulus two-response conditional discrimination task with auditory stimuli and induced response bias through unequal reinforcement probabilities for the two responses. We compared three signal-detection-theory-based criterion learning models with respect to their ability to fit experimentally observed fluctuations of response bias on a trial-by-trial level. These models shift the criterion by a fixed step (1) after each reinforced response or (2) after each nonreinforced response or (3) after both. We find that all three models fail to capture essential aspects of the data. Prompted by the observation that steady-state criterion values conformed well to a behavioral model of signal detection based on the generalized matching law, we constructed a trial-based version of this model and find that it provides a superior account of response bias fluctuations under changing reinforcement contingencies.

Effects of Episodic Memory Recall on Reinforcing Value of Physical Activity.

Many people prefer a sedentary versus an active lifestyle and have difficulty completing sufficient physical activity to improve health. While exercise can be a powerful reinforcer for some people, many prefer to be sedentary. The relative reinforcing value of physical activity (RRVPA) is influenced by a person's learning history with positive experiences strengthening the motivation to be active. Episodic memories may be critical to determining RRVPA, or RRV of different types of positive behaviors. As a potential way to increase RRVPA we examined if recalling a positive episodic memory would affect RRVPA. One hundred and thirty seven adult participants completed measures of reinforcement history and memory functions and then recalled a single positive episode of either a physical activity or of a cognitively enriching activity, rated subjective memory qualities, and completed tasks to measure the RRV of both types of activity. Recalling a positive episodic memory increased RRV for the respective activity, and changes in RRVPA were moderated by how much the memory satisfies a basic psychological need. This research demonstrates the impact of episodic memory on RRVPA, which could be leveraged to aid behavior change efforts.

다중반응교수(MEI)가 자폐스펙트럼장애 아동의 자극 등가 관계 안에서 보이는 파생적 인트라버벌 반응과 네이밍인 택트 반응에 미치는 영향에 관한 연구*

This study was to investigate effects of the Multiple Examplar Instruction(MEI) on untaught tact as Naming and untaught Intraverbal using stimuli within equivalent relations. 4 and 5 year old children with Autism Spectrum Disorder participated in the study. A multiple baselines across participants with multiple probes design was used. Direct instruction on part of possible stimulus-response relations with the stimulus equivalence relations were delivered in order to establish reinforcement history within that relations. Probe for untaught stimulus-response within the stimulus equivalence relations, derived intraverbal and derived tact as Naming were conducted. The participant showed low level of correct derived responses with the equivalence relations. A MEI was implemented where the participant learned to respond within the equivalence relations a type of relational frame. The participants showed derived intraverbal and tact responses within the frame and generalized the responding to a novel set of stimuli. Possible source for inconsistent responding across stimuli were discussed in term of behavioral cusps and stimulus control.

Why do distractions sometimes aid self-control? Pigeons (Columba livia) highlight possible mechanisms underlying the distraction effect.

In this essay, the author explores the question of why distractions sometimes aid self-control. In a study with chimpanzees, Evans and Beran (2007) used two conditions with toys to address the possibility raised by Mueller et al. (2023) about toys as distractors. In the first, the accumulating rewards were within reach, and so chimpanzees had to inhibit taking rewards if more were to accumulate. The second condition was essential to this issue, as in that case toys also were available, but the delayed reward was out of reach (i.e., the chimpanzees were forced to wait to get the delayed reward). Because these trial types were intermixed, an explanation of the toys' effectiveness through conditioned association with the delayed reward would predict that the chimpanzees should have engaged the toys equally often in both conditions. However, three of four chimpanzees engaged the toys more when the accumulating reward was within reach compared to when it was not. Evans and Beran suggested that engagement with distractors in that study occurred when it was functionally effective in aiding delayed gratification, not solely as a result of toys being associated with delayed rewards, and thus was a cognitively controlled decision by the chimpanzees. The results of Evans and Beran (2007) have yet to be replicated in any other primate species. But, Mueller et al. (2023) noted that a study with a parrot (Koepke et al., 2015) showed that the parrot would use distractors and even move the smaller, sooner reward away from itself, perhaps matching the self-distraction techniques of children who hide their faces or talk to themselves. Thus, it remains to be determined whether those behaviors also can be explained by learned associations with reward or whether they reflect attention-based explanations. Mueller et al. (2023) have provided a creative experiment and a compelling argument that more careful analysis is needed of exactly what happens when an animal (or child) engages distractors and shows improved self-control, so that we can understand what role past reinforcement history may play and what possible attentional strategies or other cognitive strategies are at work in different self-control tasks. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

An assessment of token value and effectiveness: A replication.

The purpose of the current study was to conduct a systematic replication of Fiske et al. (2015) to extend the behavior-analytic literature on token reinforcement. Specifically, the researchers addressed several of the limitations of Fiske et al. by including specific conditioning procedures, creating a controlled history of reinforcement with the token system, including participants with and without disabilities to extend the generality of the findings, and equating the magnitude of reinforcement across the primary and paired-token conditions. The current study evaluated the reinforcing value of tokens by using progressive-ratio schedules to compare the efficacy of primary reinforcement, paired tokens, and unpaired tokens. The results suggest that paired tokens may function as reinforcers, but they were not as consistently effective as primary reinforcers.

In the absence of extensive initial training, cleaner wrasse Labroides dimidiatus fail a transitive inference task.

Transitive inference (TI) is a reasoning capacity that allows individuals to deduce unknown pair relationships from previous knowledge of other pair relationships. Its occurrence in a wide range of animals, including insects, has been linked to their ecological needs. Thus, TI should be absent in species that do not rely on such inferences in their natural lives. We hypothesized that the latter applies to the cleaner wrasse Labroides dimidiatus and tested this with 19 individuals using a five-term series (A > B > C > D > E) experiment. Cleaners first learned to prefer a food-rewarding plate (+) over a non-rewarding plate (-) in four plate pairs that imply a hierarchy from plate A to plate E (A+B-, B+C-, C+D-, D+E-), with the learning order counterbalanced between subjects. We then tested for spontaneous preferences in the unknown pairs BD (transitive inference task) and AE (as a control for anchors), interspersed between trials involving a mix of all known adjacent pairs. The cleaners systematically preferred A over E and showed good performance for A+B- and D+E- trials. Conversely, cleaners did not prefer B over D. These results were unaffected by the reinforcement history, but the order of learning of the different pairs of plates had a main impact on the remembrance of the initial training pairs. Overall, cleaners performed randomly in B+C- and C+D- trials. Thus, a memory constraint may have prevented subjects from applying TI. Indeed, a parallel study on cleaner wrasse provided positive evidence for TI but was achieved following extensive training on the non-adjacent pairs which may have over-ridden the ecological relevance of the task.

Recruitment of neuronal ensembles linked to compulsive ethanol seeking in rats with a history of negative reinforcement by ethanol

Recruitment of neuronal ensembles linked to compulsive ethanol seeking in rats with a history of negative reinforcement by ethanol

Further evaluation of prompt density, rate of reinforcement, and the persistence of manding

AbstractPrevious research has evaluated the effects of prompt rates on the rate of communicative behavior. More recent research has suggested that dense prompting can result in communicative behavior that is more resistant to change. However, existing research has not considered the impact that higher response rates had on reinforcement rate, a variable known to impact response persistence. The current study systematically replicated previous research by evaluating communicative responding in contexts associated with dense‐ and lean‐prompt schedules and extended existing research by (a) holding reinforcement rates similar across the two prompting schedules (lean and dense), and (b) evaluating the persistence of communicative responding in the contexts associated with each prompting schedule. The results for Experiment 1 clearly replicated and extended previous research. The results for Experiment 2 were equivocal and suggested that previous reinforcement history and response class size impacted outcomes.

Effect of lysergic acid diethylamide (LSD) on reinforcement learning in humans.

The non-selective serotonin 2A (5-HT2A) receptor agonist lysergic acid diethylamide (LSD) holds promise as a treatment for some psychiatric disorders. Psychedelic drugs such as LSD have been suggested to have therapeutic actions through their effects on learning. The behavioural effects of LSD in humans, however, remain incompletely understood. Here we examined how LSD affects probabilistic reversal learning (PRL) in healthy humans. Healthy volunteers received intravenous LSD (75 μg in 10 mL saline) or placebo (10 mL saline) in a within-subjects design and completed a PRL task. Participants had to learn through trial and error which of three stimuli was rewarded most of the time, and these contingencies switched in a reversal phase. Computational models of reinforcement learning (RL) were fitted to the behavioural data to assess how LSD affected the updating ('learning rates') and deployment of value representations ('reinforcement sensitivity') during choice, as well as 'stimulus stickiness' (choice repetition irrespective of reinforcement history). Raw data measures assessing sensitivity to immediate feedback ('win-stay' and 'lose-shift' probabilities) were unaffected, whereas LSD increased the impact of the strength of initial learning on perseveration. Computational modelling revealed that the most pronounced effect of LSD was the enhancement of the reward learning rate. The punishment learning rate was also elevated. Stimulus stickiness was decreased by LSD, reflecting heightened exploration. Reinforcement sensitivity differed by phase. Increased RL rates suggest LSD induced a state of heightened plasticity. These results indicate a potential mechanism through which revision of maladaptive associations could occur in the clinical application of LSD.

Assessing lithological uncertainty in dikes: Simulating construction history and its implications for flood safety assessment

AbstractDikes often have a long history of reinforcement, with each reinforcement adding new material resulting in a heterogeneous dike. As data on the dike internal heterogeneity is sparse, it is generally overlooked in the stability assessment of dikes. We present an object‐based and process‐based model simulating dike construction history on archeological dike cross, yielding similar patterns of heterogeneity as observed in real dikes, and apply it in a dike safety assessment. Model predictions improve when being based on more accurate statistics of dike buildup, or when being conditioned to ground truth data. When incorporated in a dike stability assessment, multiple model runs can be coupled to hydrological simulations and dike slope stability calculations, resulting in a probabilistic stability assessment considering internal dike heterogeneity. While high‐resolution observations are still sparse, good model accuracies can be reached by combining regional information on dike buildup with local point observations and this model provides a parsimonious basis to include information of internal dike heterogeneity in safety assessments.

Impact of previous reinforcement on false perceptions for individuals lower and higher in schizotypy traits

Psychotic-like experiences (hallucinations) were investigated in a non-clinical population in a word-recognition experiment. The current study explored the effect of perceived importance (previous reinforcement), and whether this differs in its impact on individuals with lower or higher schizotypy. Participants were assessed psychometrically for their level of schizotypy using the O-LIFE, and were reinforced for choosing wither nouns or verbs. A word recognition task was then presented, during which words or non-words were presented on a fast-moving display was employed. The words could wither be nouns or verbs. Those higher in schizotypy showed greater numbers of false perceptions than those with lower schizotypy, and previous reinforcement history impacted on performance. When words from a previously reinforced class were tested, recognition of words that were present was higher than when the test class was not previously reinforced. Moreover, in conditions where expectancies had been violated, those with higher schizotypy showed greater numbers of false perceptions than those with lower schizotypy scores. Thus, the current findings show situational factors such as previous experience and current context are also important in generating false perceptions.

다중반응교수법(MEI)이 발달장애 아동의 자극 등가 관계 안에서 보이는 파생적 읽기 반응과 수 개념 반응에 미치는 영향에 관한 연구

The purpose of this study was to investigate whether the Multiple Examplar Instruction(MEI) induced untaught reading responses and number concept responses. These untaught target responses were induced within a stimulus class which showed stimulus equivalence relation. A participant was a first grader with developmental disabilities in an elementary school. The participant exhibited developmental disabilities including language delays. She tacted over 400 objects and pictures, placed number cards in a right order. She worked on early reading skills and number skills. 3 types of stimulus were used for reading responses and number concept responses during MEI and pre- and post-probes: picture cards, corresponding written words, words read vocally for reading and number cards, picture cards with an amount of a number shown and spoken ordinal numbers for number concepts. A multiple probe design was used in this study. First, direct instruction on some of possible stimulus-response relations within the stimulus equivalence relations were delivered in order to establish reinforcement history within that relations. Then, probe for untaught stimulus-response within the same stimulus equivalence relations were conducted. The participant showed low level of correct derived responses with the equivalence relations. A MEI was implemented where the participant learned to respond within the equivalence relations as a type of relational frame. The Participant showed target derived responses within the stimulus equivalence relations during the Post-MEI probe and generalized the derived responding to a novel set of stimuli.

History and Development of Polymeric Denture Base Reinforcement

History and Development of Polymeric Denture Base Reinforcement

Machine learning with a snapshot of data: Spiking neural network 'predicts' reinforcement histories of pigeons' choice behavior.

An accumulated body of choice research has demonstrated that choice behavior can be understood within the context of its history of reinforcement by measuring response patterns. Traditionally, work on predicting choice behaviors has been based on the relationship between the history of reinforcement—the reinforcer arrangement used in training conditions—and choice behavior. We suggest an alternative method that treats the reinforcement history as unknown and focuses only on operant choices to accurately predict (more precisely, retrodict) reinforcement histories. We trained machine learning models known as artificial spiking neural networks (SNNs) on previously published pigeon datasets to detect patterns in choices with specific reinforcement histories—seven arranged concurrent variable‐interval schedules in effect for nine reinforcers. Notably, SNN extracted information from a small ‘window’ of observational data to predict reinforcer arrangements. The models' generalization ability was then tested with new choices of the same pigeons to predict the type of schedule used in training. We examined whether the amount of the data provided affected the prediction accuracy and our results demonstrated that choices made by the pigeons immediately after the delivery of reinforcers provided sufficient information for the model to determine the reinforcement history. These results support the idea that SNNs can process small sets of behavioral data for pattern detection, when the reinforcement history is unknown. This novel approach can influence our decisions to determine appropriate interventions; it can be a valuable addition to our toolbox, for both therapy design and research.

Reinforcement History Dependent Effects of Low Dose Ethanol on Reward Motivation in Male and Female Mice.

Alcohol use disorders (AUDs) are more prevalent in men than in women, though AUD diagnoses in women are growing rapidly, making an understanding of sex differences in alcohol-related behaviors increasingly important. The development of AUDs involves the transition from casual, low levels of alcohol drinking to higher, maladaptive levels. The ability of low dose alcohol to drive reward and drug seeking may differ in males and females, and this could underlie differences in susceptibility to AUD. In this study we sought to determine whether a history of chronic, low dose ethanol exposure (0.5 g/kg; i.p.) could drive sucrose reward seeking and motivation, and whether this differed between male and female mice. Adult mice were trained to lever press for a liquid sucrose reward on two reinforcement schedules: a random interval (RI) schedule and a variable ratio (VR) schedule. After training, mice were tested on each of these levers for reward motivation using a progressive ratio test. We found that a history of low dose ethanol exposure increased sucrose reward motivation in male mice, but only on the RI lever and only when exposure occurred proximal to learning. Female mice were more motivated for sucrose on the RI lever than the VR lever regardless of ethanol exposure condition. These findings indicate that training on different reinforcement schedules affects reward motivation. Further, we show that males are more susceptible to the effects of low dose ethanol on sucrose reward motivation than females. These data broaden our understanding of sex differences in reward seeking as a result of ethanol exposure.