Nonreinforced Trials Research Articles

Honey bees rely on associative stimulus strength after training on an olfactory transitive inference task.

Transitive inference, the ability to establish hierarchical relationships between stimuli, is typically tested by training with premise pairs (e.g., A + B-, B + C-, C + D-, D + E-), which establishes a stimulus hierarchy (A > B > C > D > E). When subjects are tested with non-adjacent stimuli (e.g., B vs. D), a preference for B indicates transitive inference, while no preference indicates decisions based on stimulus associative strength, as B and D are equally reinforced. Previous studies with bees and wasps, conducted in an operant context, have shown conflicting results. However, this context allows free movement and the possibility to avoid non-reinforced options, thus reducing the number of non-reinforced trials. To address this, we examined whether honey bees could perform transitive inference using a Pavlovian protocol that fully controls reinforcement. We conditioned bees with five odorants, either forward-or backward-paired with a sucrose solution, across four discrimination tasks. In all experiments, bees showed no preference for B over D, choosing equally between them, regardless of the training schedule. Our results show that bees' choices were primarily influenced by stimulus associative strength and a recency effect, with greater weight given to the most recent reinforced or non-reinforced stimulus. We discuss these findings in the context of honey bee memory, suggesting that memory constraints may limit cognitive solutions to transitive inference tasks in bees.

Variation in the effectiveness of reinforcement and nonreinforcement in generating different conditioned behaviors

Rat autoshaping procedures generate two readily measurable conditioned responses: During lever presentations that have previously signaled food, rats approach the food well (called goal-tracking) and interact with the lever itself (called sign-tracking). We investigated how reinforced and nonreinforced trials affect the overall and temporal distributions of these two responses across 10-second lever presentations.In two experiments, reinforced trials generated more goal-tracking than sign-tracking, and nonreinforced trials resulted in a larger reduction in goal-tracking than sign-tracking. The effect of reinforced trials was evident as an increase in goal-tracking and reduction in sign-tracking across the duration of the lever presentations, and nonreinforced trials resulted in this pattern transiently reversing and then becoming less evident with further training.These dissociations are consistent with a recent elaboration of the Rescorla-Wagner model, HeiDI (Honey, R.C., Dwyer, D.M., & Iliescu, A.F. (2020a). HeiDI: A model for Pavlovian learning and performance with reciprocal associations. Psychological Review, 127, 829–852.), a model in which responses related to the nature of the unconditioned stimulus (e.g., goal-tracking) have a different origin than those related to the nature of the conditioned stimulus (e.g., sign-tracking).

Learning about trial sequences disrupts the partial reinforcement extinction effect in classical conditioning.

The partial reinforcement extinction effect (PREE) refers to the phenomenon that conditioned responding extinguishes more slowly if subjects had been inconsistently ("partially") reinforced than if they had been reinforced on every trial ("continuously" reinforced). One largely successful account of the PREE, known as sequential theory (Capaldi, 1966), suggests that, when subjects are partially reinforced, they learn that memories of sequences of nonreinforced trials are associated with subsequent reinforcement. This association helps to maintain responding (i.e., delay extinction) when the subjects experience nonreinforced trials during extinction. Sequential theory's explanation of the PREE hinges on subjects learning sequences of nonreinforced trials during acquisition. However, direct evidence for such sequential learning is not available in previous studies of the PREE where animals are trained with multiple sequences of different lengths that are randomly intermixed and, therefore, cannot anticipate whether a given trial will be reinforced during acquisition. The current study conducted two experiments that trained rats with a single fixed trial sequence to provide evidence of sequential learning during conditioning, and then observe its effect on the PREE. Under one condition the rats did learn about the fixed sequence but did not subsequently show a PREE, whereas other rats that did show a PREE had not learned the trial sequences during conditioning. Therefore, contrary to sequential theory's prediction, our result suggests that learning about the trial sequence is neither necessary nor sufficient for the PREE. We suggest that the PREE may instead depend on uncertainty about whether the conditioned stimulus will be reinforced. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Signal detection analysis of contingency assessment: Associative interference and nonreinforcement impact cue-outcome contingency sensitivity, whereas cue density affects bias.

In a signal detection theory approach to associative learning, the perceived (i.e., subjective) contingency between a cue and an outcome is a random variable drawn from a Gaussian distribution. At the end of the sequence, participants report a positive cue-outcome contingency provided the subjective contingency is above some threshold. Some researchers have suggested that the mean of the subjective contingency distributions and the threshold are controlled by different variables. The present data provide empirical support for this claim. In three experiments, participants were exposed to rapid streams of trials at the end of which they had to indicate whether a target outcome O1 was more likely following a target cue X. Interfering treatments were incorporated in some streams to impend participants' ability to identify the objective X-O1 contingency: interference trials (X was paired with an irrelevant outcome O2), nonreinforced trials (X was presented alone), plus control trials (an irrelevant cue W was paired with O2). Overall, both interference and nonreinforced trials impaired participants' sensitivity to the contingencies as measured by signal detection theory's d', but they also enhanced detection of positive contingencies through a cue density effect, with nonreinforced trials being more susceptible to this effect than interference trials. These results are explicable if one assumes interference and nonreinforced trials impact the mean of the associative strength distribution, while the cue density influences the threshold. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Partial reinforcement effects on acquisition and extinction of a conditioned taste aversion.

Four experiments with rat subjects asked whether a partial reinforcement extinction effect (PREE) occurs in taste aversion learning. The question has received little attention in the literature, and to our knowledge no taste aversion experiment has previously demonstrated a PREE. In each of the present experiments, experimental groups received a taste mixed in drinking water for 20 min; such taste exposures were sometimes paired with a lithium chloride (LiCl) injection and sometimes not. Control groups received only taste-LiCl pairings. There was evidence that each reinforced and non-reinforced trial produced increments and decrements in aversion strength (respectively), and trials mattered more than accumulated time during the conditioned stimulus and during the background (as emphasized in time-accumulation models like those of Gallistel and Gibbon, Psychological Review, 107, 289-344, 2000, and Gibbon and Balsam, Autoshaping and conditioning theory, Academic Press, New York, pp. 219-235, 1981). In addition, a partial reinforcement extinction effect was observed when there was a relatively large number of conditioning trials. The results extend our understanding of extinction in taste aversion learning and provide more evidence that aversion learning might follow rules that are qualitatively similar to those of other forms of learning.

Assessing human performance during contingency changes and extinction tests in reversal-learning tasks.

Serial reversal-learning procedures are simple preparations that allow for a better understanding of how animals learn about environmental changes, including flexibly shifting responding to adapt to changing reinforcement contingencies. The present study examined serial reversal learning with humans by arranging both midsession and variable contingency reversals across two experiments. We also examined the effects of extinction by adding nonreinforced trials at the end of later sessions and provided the first evaluation of effects of win-stay/lose-shift versus counting strategies on accuracy and response latency of humans' reversal-learning performance. In each experiment, responding tracked contingency reversals, primarily with participants using either win-stay/lose-shift or counting strategies. Introducing variable reversal points in the second experiment resulted in near-exclusive win-stay/lose-shift responding among participants and eliminated counting of trials. Each experiment also revealed an immediate shift from S2 to S1 after experiencing extinction during the initial test trial, indicating resurgence of the initial response through a win-stay/lose-shift response pattern. Therefore, the present study replicates and extends prior findings of a win-stay/lose shift response pattern in situations of greater uncertainty. These findings suggest that differences in environmental certainty induce qualitatively different decision-making strategies.

Assessing the inhibitory properties of a latent inhibitor in flavor-aversion learning

In Experiment 1, rats received 16 nonreinforced trials of exposure to a flavor (A) that was subsequently used as the conditioned stimulus in flavor-aversion conditioning. In the critical condition, Flavor A was presented in compound with a different novel flavor on each of the eight daily trials. This treatment produced latent inhibition, in that this preexposure retarded conditioning just as did 16 trials with A alone. Rats in the control conditions, given no preexposure or exposure just to the sequence of novel flavors, learned readily. Experiment 2 examined the effects of these forms of preexposure on performance on a summation test, in which Flavor A was presented in compound with a separately conditioned flavor (X). The preexposure procedure in which A was presented along with novel flavors rendered A effective in inhibiting the response conditioned to X on that test. The conclusion, that this form of training can establish the target stimulus as a conditioned inhibitor, is predicted by the account of latent inhibition put forward by Hall and Rodríguez (2010) which proposes that the latent inhibition effect is a consequence both of a reduction in the associability of the stimulus and of a process of inhibitory associative learning that opposes the initial expectation that a novel event will be followed by some consequence.

Effects of cycloheximide on recent and remote appetitive odor discrimination memory in rats

A variety of reconsolidation boundary conditions exist for the destabilization of memory, including memory strength, prediction error, trace dominance, and memory age. The vast majority of studies examining boundary conditions have employed aversive tasks. The current study expands the literature on reconsolidation boundary conditions by employing an appetitive odor discrimination task and a delay that exceeds those used in past studies (i.e., 72 days). Rats were trained to dig in cups of scented sand to retrieve a sweet cereal reward. One day (Recent) or 72 days (Remote) following acquisition, rats received a single reactivation trial that included a prediction error (in the form of a non-reinforced trial), followed by intraperitoneal administration of the protein synthesis inhibitor cycloheximide (CHX; 1 mg/kg) or a saline control (SAL). One week later, rats completed a single non-reinforced test trial. Results showed that rats in the Remote condition took longer to dig than those in the Recent condition; additionally, CHX disrupted memory for digging behavior at significant levels in both the Recent and Remote conditions. Results are consistent with a small number of studies using non-aversive tasks, as well as a large number of studies using aversive tasks, demonstrating that recent and remote memories are vulnerable to the effects of an amnesic agent after sufficient reactivation.

Pavlovian conditioning under partial reinforcement: The effects of nonreinforced trials versus cumulative conditioned stimulus duration.

A core feature of associative models, such as those proposed by Allan Wagner (Rescorla & Wagner, 1972; Wagner, 1981), is that conditioning proceeds in a trial-by-trial fashion, with increments and decrements in associative strength occurring on each occasion that the conditioned stimulus (conditional stimulus, or CS) is present either with or without the unconditioned stimulus (US). A very different approach has been taken by theories that assume animals continuously accumulate information about the total length of time spent waiting for the US both during the CS and in the absence of the CS (e.g., Gallistel & Gibbon, 2000). Here we describe 3 experiments using within-subject designs that tested trial-based and time-accumulation accounts of the acquisition of conditioned responding using magazine approach conditioning in rats. We found that responding was affected by the total (cumulative) duration of exposure to the CS without the US rather than the number of trials on which the CS occurred without the US. We also found that exposure to the CS without the US had the same effect on conditioning whether that exposure occurred shortly (60 s) before each CS-US pairing or whether it occurred long (240 s) before each pairing. These findings are more consistent with time-accumulation models of conditioning than trial-based models like the Rescorla-Wagner model and Wagner's (1981) sometimes opponent process model. We discuss these findings in relation to other evidence that favors trial-based models rather than time-accumulation models. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Summation and retardation test performance following extinction or Pavlovian conditioned inhibition training

There is evidence that both Pavlovian conditioned inhibition training and CS-alone extinction trials result in a CS that passes a summation and retardation test. The present experiments compared the extent that these two procedures produced a CS that could pass these tests. Such a comparison could provide insight into the kind of inhibitory learning that results, specifically whether two CSs on the nonreinforced trials compete for the acquisition of inhibitory learning. Very little evidence of competition for inhibitory learning has been obtained in previously published studies. Evidence consistent with more inhibition occurring using the Pavlovian CI procedure than extinction was obtained.

P.813 MK-801 blocks the reconsolidation of memory reactivated by a single non-reinforced trial and subsequent reverse learning in 8-arm radial maze

P.813 MK-801 blocks the reconsolidation of memory reactivated by a single non-reinforced trial and subsequent reverse learning in 8-arm radial maze

The partial reinforcement extinction effect depends on learning about nonreinforced trials rather than reinforcement rate.

Conditioned responding extinguishes more slowly after partial (inconsistent) reinforcement than after consistent reinforcement. This Partial Reinforcement Extinction Effect (PREE) is usually attributed to learning about nonreinforcement during the partial schedule. An alternative explanation attributes it to any difference in the rate of reinforcement, arguing that animals can detect the change to nonreinforcement more quickly after a denser schedule than a leaner schedule. Experiments 1a and 1b compared extinction of magazine responding to a conditioned stimulus (CS) reinforced with 1 food pellet per trial and a CS reinforced with 2 pellets per trial. Despite the difference in reinforcement rate, there was no reliable difference in extinction. Both experiments did demonstrate the conventional PREE comparing a partial CS (50% reinforced) with a consistent CS. Experiments 2 and 3 tested whether the PREE depends specifically on learning about nonreinforced trials during partial reinforcement. Rats were trained with 2 CS configurations, A and AX. One was partially reinforced, the other consistently reinforced. When AX was partial and A consistent, responding to AX extinguished more slowly than to A. When AX was consistent and A was partial, there was no difference in their extinction. Therefore, pairing X with partial reinforcement allowed rats to show a PREE to AX that did not generalize to A. Pairing A with partial reinforcement meant that rats showed a PREE to A that generalized to AX. Thus, the PREE depends on learning about nonreinforced trials during partial reinforcement and is not because of any difference in per-trial probability of reinforcement. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The role of prediction error and memory destabilization in extinction of cued-fear within the reconsolidation window

Extinction of a cued-fear memory within the reconsolidation window has been proposed to prevent fear reacquisition by reconsolidation interference. This ‘retrieval-extinction’ procedure has received interest for its therapeutic potential to reduce the impact of fear memories on behavior. To fully exploit its therapeutic potential, it is critical to understand the mechanisms that underlie the ‘retrieval-extinction’ effect. If the effect depends upon reconsolidation of the original memory, then it would be predicted that destabilization, induced by prediction error, would be critical for observing the effect. Here, the dependency of the retrieval-extinction effect on memory destabilization or prediction error was investigated in pavlovian cued-fear conditioned adult male rats. The requirement for memory destabilization, and thus reconsolidation, for the retrieval-extinction effect was subsequently investigated using region-specific pharmacological blockade of dopamine D1-receptors. Intra-basolateral amygdala antagonism of dopamine D1-receptors did not prevent the reacquisition of fear associated with the retrieval-extinction procedure. The requirement for prediction error was assessed by using a reinforced or non-reinforced memory retrieval trial before extinction, compared to a no-retrieval, extinction-only control. Both the reinforced (no prediction error) and non-reinforced retrieval sessions led to a decrease in fear reacquisition, suggesting that engagement of prediction error does not influence the occurrence of retrieval-extinction. Together, these data suggest that retrieval-extinction does not require memory destabilization, since behavioral or pharmacological interventions that prevent destabilization did not disrupt any capacity to attenuate fear.

Associative structure of second-order conditioning in humans.

Second-order conditioning (SOC; i.e., conditioned responding to S2 as a result of S1-US pairings followed by S2-S1 pairings) is generally explained by either a direct S2→US association or by an associative chain (i.e., S2→S1→US). Previous research found that differences in responses to S2 after S1 was extinguished often depended on the nature of the S2-S1 pairings (i.e., sequential or simultaneous). In two experiments with human participants, we examined the possibility that such differences result from S1 evoking S2 during extinction of S1 following simultaneous but not sequential S2-S1 pairings. This evocation of S2 by S1 following simultaneous pairings may have paired the evoked representation of S2 with absence of the outcome, thereby facilitating mediated extinction of S2. Using sequential S2-S1 pairings, both Experiments 1 and 2 failed to support this account of how extinction of S1 reduced responding to S2. Experiment 1 found that extinguishing S1 reduced responding to S2, while extinguishing S2 had little effect on responses to S1, although forward evocation of S1 during extinction of S2 paired the evoked representation of S1 with absence of the outcome. In Experiment 2, evocation of S2 during S1 nonreinforced trials was prevented because S2-S1 pairings followed (rather than proceeded) S1-alone exposures. Nevertheless, responding to S2 at test mimicked S1 responding. Responding to S2 was high in the context in which S1 had been reinforced and low in the context in which S1 had been nonreinforced. Collectively, these experiments provide additional support for the associative-chain account of SOC.

The relationship between behavior acquisition and persistence abilities: Involvement of adult hippocampal neurogenesis.

The influence of the learning process on the persistence of the newly acquired behavior is relevant both for our knowledge of the learning/memory mechanisms and for the educational policy. However, it is unclear whether during an operant conditioning process with a continuous reinforcement paradigm, individual differences in acquisition are also associated to differences in persistence of the acquired behavior. In parallel, adult neurogenesis has been implicated in spatial learning and memory, but the specific role of the immature neurons born in the adult brain is not well known for this process. We have addressed both questions by analyzing the relationship between water maze task acquisition scores, the persistence of the acquired behavior, and the size of the different subpopulations of immature neurons in the adult murine hippocampus. We have found that task acquisition and persistence rates were negatively correlated: the faster the animals find the water maze platform at the end of acquisition stage, the less they persist in searching for it at the learned position in a subsequent non-reinforced trial; accordingly, the correlation in the number of some new neurons' subpopulations and the acquisition rate is negative while with persistence in acquired behavior is positive. These findings reveal an unexpected relationship between the efficiency to learn a task and the persistence of the new behavior after a non-reinforcement paradigm, and suggest that the immature neurons might be involved in different roles in acquisition and persistence/extinction of a learning task. © 2016 Wiley Periodicals, Inc.

Characterizing Olfactory Binary Mixture Interactions in Fischer 344 Rats Using Behavioral Reaction Times

Response times provide essential subthreshold perceptual data that extend beyond accuracy alone. Behavioral reaction times (RTs) were used to characterize rats' ability to detect individual odorants in a series of complimentary binary odorant mixture ratios. We employed an automated, liquid-dilution olfactometer to train Fischer 344 rats (N = 8) on an odor identification task using nonreinforced probe trials. Binary mixture ratios composed of aliphatic odorants (citral and octanol) were arranged such that relative contributions of the 2 components varied systematically by a factor of 1% (v/v). Odorant concentrations for the target (S+), control (S-), and mixture (S+:S-) odorants were presented relative to threshold for each rat. Rats were initially trained to respond by licking at a spout to obtain liquid reward for either citral or octanol as the reinforced target (S+) odorant. After achieving 100% accuracy, rats were transferred to variable ratio (VR 2) reinforcement for correct responding. Nonreinforced probe trials (2 per block of 22 trials) were tested for each mixture ratio and recorded as either S+ (rats lick-responded in the presence of the mixture) or S- (rats refrained from licking), thereby indicating detection of the trained, S+ odorant. To determine the perceived salience for each ratio, RTs (latency from odorant onset to lick response) were recorded for each trial. Consistent with previous studies, RTs for both odorants were shortest (~150-200ms) when the probe trials consisted of a single, monomolecular component. Binary mixtures that contained as little as 1% of the S-, nontarget odorant, however, were sufficiently different perceptually to increase behavioral RTs (i.e., rats hesitated longer before responding); RTs changed systematically as a function of the binary ratio. Interestingly, the rate of RT change was dependent on which odorant served as the S+, suggesting an asymmetric interaction between the 2 odorants. The data demonstrate the value of behavioral RT as a sensitive measure of suprathreshold perceptual responding.

Contribution of the retrosplenial cortex to temporal discrimination learning.

The retrosplenial cortex (RSC) has an important role in contextual learning and memory. While the majority of experiments have focused on the physical context, the present study asked whether the RSC is involved in processing the temporal context. Rats were trained in a temporal discrimination procedure where the duration of the intertrial interval (ITI) signaled whether or not the next tone conditioned stimulus would be paired with food pellet reinforcement. When the tone was presented after a 16-min ITI it was reinforced, but when it was presented after a 4-min ITI it was not. Rats demonstrated successful discrimination in this procedure by responding more to the tone on reinforced trials than on non-reinforced trials. Pre-training electrolytic lesions of the RSC attenuated acquisition of the temporal discrimination. The results are the first to demonstrate a role for the RSC in processing temporal information and in turn extend the role of the RSC beyond the physical context to now include the temporal context.

Inhibitory learning is modulated by nicotinic acetylcholine receptors

Prior research has established that stimulating nicotinic acetylcholine receptors can facilitate learning and memory. However, most studies have focused on learning to emit a particular behavior, while little is known about the effects of nicotine on learning to withhold a behavioral response. The present study consisted of a dose response analysis of the effects of nicotine on negative occasion setting, a form of learned inhibition. In this paradigm, rats received one type of training trial in which presentation of a tone by itself was followed immediately by food reward. During the other type of trials, the tone was preceded by presentation of a light and no food was delivered after the tone. Rats gradually learned to approach the cup in anticipation of receiving food reward during presentations of the tone alone, but withheld that behavior when the tone was preceded by the light. Nicotine (0.35 mg/kg) facilitated negative occasion setting by reducing the number of sessions needed to learn the discrimination between trial types and by reducing the rate of responding on non-reinforced trials. Nicotine also increased the orienting response to the light, suggesting that nicotine may have affected the ability to withhold food cup behavior on non-reinforced trials by increasing attention to the light. In contrast to the effects of nicotine, rats treated with mecamylamine (0.125, 0.5, or 2 mg/kg) needed more training sessions to discriminate between reinforced and non-reinforced trials compared to saline-treated rats. The findings indicate that nicotinic acetylcholine receptors may be active during negative occasion setting and that nicotine can potentiate learned inhibition.

The ontogeny of learned inhibition

Previous studies have examined the maturation of learning and memory abilities during early stages of development. By comparison, much less is known about the ontogeny of learning and memory during later stages of development, including adolescence. In Experiment 1, we tested the ability of adolescent and adult rats to learn a Pavlovian negative occasion setting task. This procedure involves learning to inhibit a behavioral response when signaled by a cue in the environment. During reinforced trials, a target stimulus (a tone) was presented and immediately followed by a food reward. On nonreinforced trials, a feature stimulus (a light) was presented 5 sec prior to the tone and indicated the absence of reward following presentation of the tone. Both adult and adolescent rats learned to discriminate between two different trial types and withhold responding when the light preceded the tone. However, adolescent rats required more sessions than adults to discriminate between reinforced and nonreinforced trials. The results of Experiment 2 revealed that adolescents could learn the task rules but were specifically impaired in expressing that learning in the form of withholding behavior on nonreinforced trials. In Experiment 3, we found that adolescents were also impaired in learning a different version of the task in which the light and tone were presented simultaneously during the nonreinforced trials. These findings add to existing literature by indicating that impairments in inhibitory behavior during adolescence do not reflect an inability to learn to inhibit a response, but instead reflect a specific deficit in expressing that learning.

Transient decline in rats’ hippocampal theta power relates to inhibitory stimulus-reward association

The hippocampus is important in learning during a discrimination-reversal task. In this task, animals first learn to emit the go response to one stimulus and the no-go response to another stimulus (S1+, S2−) during the discrimination phase, and then they learn to reverse these relationships between stimulus and response during the reversal phase (S1−, S2+). To emit a no-go response for non-reinforced trial during the reversal phase, animals needed to inhibit the previously learned response pattern. This study examined the relationship between the reversal phase of the discrimination-reversal task and hippocampal electric activity in operant conditioning. The results revealed that hippocampal theta power transiently declined during the non-reinforced trial in the reversal phase compared with that during the discrimination phase. This decrease was observed during the 400–600-ms epoch after the onset of stimulus presentation. This study suggested that transient decline in hippocampal theta power is related to negative memory retrieval.