Reinforcer Research Articles

Neural correlates of a reversal learning task with an affectively neutral baseline: An event-related fMRI study

Reversal learning may conceptually be dissected into acquiring stimulus–reinforcement associations and subsequently altering behavior by switching to new associations as stimulus–reinforcement contingencies reverse (i.e., affective switching). Previous imaging studies have found regions of the ventrolateral and orbitofrontal cortex (OFC) to be involved in both subprocesses. However, these studies did not contain an affectively neutral baseline, which precluded adequate assessment of main effects of reward, punishment, and affective switching. We aimed to determine the neural substrate of these main effects, and of common and dissociable regions for reward and punishment. Furthermore, we aimed to discriminate between stimulus–punishment association and affective switching, i.e., to assess affective switching proper. To this end, we implemented a reversal learning task with an affectively neutral baseline condition that matched the experimental task in visual complexity and motor demands. Interestingly, we found dorsolateral prefrontal cortex (DLPFC) and anterior PFC to be engaged in affective switching, a finding that has not been reported before to our knowledge. Enhanced responses in these areas may represent their involvement in cognitive set shifting per se unrelated to the affective context in a reversal learning design. In addition, OFC, insular and medial prefrontal cortex regions were involved in affective switching. Left medial and lateral OFC were shown to be common areas for feedback processing, whereas left ventral striatum and left lateral OFC were specifically activated by reward and punishment, respectively. These results extend our understanding of the neural substrate of reversal learning in humans.

Associative learning shapes the neural code for stimulus magnitude in primary auditory cortex

Since the dawn of experimental psychology, researchers have sought an understanding of the fundamental relationship between the amplitude of sensory stimuli and the magnitudes of their perceptual representations. Contemporary theories support the view that magnitude is encoded by a linear increase in firing rate established in the primary afferent pathways. In the present study, we have investigated sound intensity coding in the rat primary auditory cortex (AI) and describe its plasticity by following paired stimulus reinforcement and instrumental conditioning paradigms. In trained animals, population-response strengths in AI became more strongly nonlinear with increasing stimulus intensity. Individual AI responses became selective to more restricted ranges of sound intensities and, as a population, represented a broader range of preferred sound levels. These experiments demonstrate that the representation of stimulus magnitude can be powerfully reshaped by associative learning processes and suggest that the code for sound intensity within AI can be derived from intensity-tuned neurons that change, rather than simply increase, their firing rates in proportion to increases in sound intensity.

A role for ERKII in synaptic pattern selectivity on the time‐scale of minutes

Stimulus reinforcement strengthens learning. Intervals between reinforcement affect both the kind of learning that occurs and the amount of learning. Stimuli spaced by a few minutes result in more effective learning than when massed together. There are several synaptic correlates of repeated stimuli, such as different kinds of plasticity and the amplitude of synaptic change. Here we study the role of signalling pathways in the synapse on this selectivity for spaced stimuli. Using the in vitro hippocampal slice technique we monitored long-term potentiation (LTP) amplitude in CA1 for repeated 100-Hz, 1-s tetani. We observe the highest LTP levels when the inter-tetanus interval is 5-10 min. We tested biochemical activity in the slice following the same stimuli, and found that extracellular signal-regulated kinase type II (ERKII) but not CaMKII exhibits a peak at about 10 min. When calcium influx into the slice is buffered using AM-ester calcium dyes, amplitude of the physiological and biochemical response is reduced, but the timing is not shifted. We have previously used computer simulations of synaptic signalling to predict such temporal tuning from signalling pathways. In the current study we consider feedback and feedforward models that exhibit temporal tuning consistent with our experiments. We find that a model incorporating post-stimulus build-up of PKM zeta acting upstream of mitogen-activated protein kinase is sufficient to explain the observed temporal tuning. On the basis of these combined experimental and modelling results we propose that the dynamics of PKM activation and ERKII signalling may provide a mechanism for functionally important forms of synaptic pattern selectivity.

The functions of the orbitofrontal cortex

The orbitofrontal cortex contains the secondary taste cortex, in which the reward value of taste is represented. It also contains the secondary and tertiary olfactory cortical areas, in which information about the identity and also about the reward value of odours is represented. The orbitofrontal cortex also receives information about the sight of objects from the temporal lobe cortical visual areas, and neurons in it learn and reverse the visual stimulus to which they respond when the association of the visual stimulus with a primary reinforcing stimulus (such as taste) is reversed. This is an example of stimulus–reinforcement association learning, and is a type of stimulus–stimulus association learning. More generally, the stimulus might be a visual or olfactory stimulus, and the primary (unlearned) positive or negative reinforcer a taste or touch. A somatosensory input is revealed by neurons that respond to the texture of food in the mouth, including a population that responds to the mouth feel of fat. In complementary neuroimaging studies in humans, it is being found that areas of the orbitofrontal cortex are activated by pleasant touch, by painful touch, by taste, by smell, and by more abstract reinforcers such as winning or losing money. Damage to the orbitofrontal cortex can impair the learning and reversal of stimulus–reinforcement associations, and thus the correction of behavioural responses when there are no longer appropriate because previous reinforcement contingencies change. The information which reaches the orbitofrontal cortex for these functions includes information about faces, and damage to the orbitofrontal cortex can impair face (and voice) expression identification. This evidence thus shows that the orbitofrontal cortex is involved in decoding and representing some primary reinforcers such as taste and touch; in learning and reversing associations of visual and other stimuli to these primary reinforcers; and in controlling and correcting reward-related and punishment-related behavior, and thus in emotion. The approach described here is aimed at providing a fundamental understanding of how the orbitofrontal cortex actually functions, and thus in how it is involved in motivational behavior such as feeding and drinking, in emotional behavior, and in social behavior.

Training Tammar Wallabies (Macropus eugenii) to Respond to Predators: A Review Linking Experimental Psychology to Conservation

Animals bred in captivity often suffer high levels of predation after release into the wild. Prerelease predator avoidance training has been undertaken to try to improve antipredator skills. Applied research on predator avoidance learning in birds and mammals has not benefited from the empirical findings of extensive basic research, as it has been the case for fish. Consequently, this field has progressed slowly and the utility of prerelease antipredator training as a conservation strategy remains controversial. Here, I report one experiment and review two others that illustrate the way in which principles and experimental designs borrowed from classic studies of animal learning can be used to develop predator avoidance training techniques and to establish the content of learning. Results show that tammar wallabies (Macropus eugenii), an Australian macropodid marsupial, can acquire a fear response that is specific to predators, but that the likelihood of learning is dependent upon subtle details of the training protocol. Differential reinforcement of predator and non-predator stimuli has the potential to enhance the specificity of learning, if necessary. I discuss the implications of these results for the field of predator avoidance training and suggest that a controlled experimental approach, which enables the content of learning to be described, will be the most fruitful for this research area in the long term.

Cognitive Neuroscience: implications for education?

Research into the functioning of the human brain, particularly during the past decade, has greatly enhanced our understanding of cognitive behaviours which are fundamental to education: learning, memory, intelligence, emotion. Here, we argue the case that research findings from cognitive neuroscience hold implications for educational practice. In doing so we advance a bio‐psycho‐social position that welcomes multi‐disciplinary perspectives on current educational challenges. We provide some examples of research implications which support conventional pedagogic wisdom, and others which are novel and perhaps counter‐intuitive. As an example, we take a model of adaptive plasticity that relies on stimulus reinforcement and examine possible implications for pedagogy and curriculum depth. In doing so, we reject some popular but over‐simplistic applications of neuroscience to education. In sum, the education profession could benefit from embracing rather than ignoring cognitive neuroscience. Moreover, educationists should be actively contributing to the research agenda of future brain research.

Suppression of cortical representation through backward conditioning.

Temporal stimulus reinforcement sequences have been shown to determine the directions of synaptic plasticity and behavioral learning. Here, we examined whether they also control the direction of cortical reorganization. Pairing ventral tegmental area stimulation with a sound in a backward conditioning paradigm specifically reduced representations of the paired sound in the primary auditory cortex (AI). This temporal sequence-dependent bidirectional cortical plasticity modulated by dopamine release hypothetically serves to prevent the over-representation of frequently occurring stimuli resulting from their random pairing with unrelated rewards.

Clinical Guide to Alcohol Treatment. The Community Reinforcement Approach.

The above is the first detailed text to implement the community reinforcement approach. It starts with the initial studies by Azrin and colleagues (1973), emphasizing at that time random controlled trials were rare in the treatment of alcoholism. The basis of the community reinforcement approach is ‘to utilise social recreational or familiar vocational reinforcers to aid clients in the recovery process’. Early studies showed positive results using this method compared with controls in alcoholics and the method was extended to cocaine and heroin users. A very wide range of reinforcers were used which in other programmes would be regarded as strategies; thus, for example in the community reinforcement approach with cocaine misusers, reinforcers …

Cognitive Neuroscience: implications for education?

Abstract Research into the functioning of the human brain, particularly during the past decade, has greatly enhanced our understanding of cognitive behaviours which are fundamental to education: learning, memory, intelligence, emotion. Here, we argue the case that research findings from cognitive neuroscience hold implications for educational practice. In doing so we advance a bio‐psycho‐social position that welcomes multi‐disciplinary perspectives on current educational challenges. We provide some examples of research implications which support conventional pedagogic wisdom, and others which are novel and perhaps counter‐intuitive. As an example, we take a model of adaptive plasticity that relies on stimulus reinforcement and examine possible implications for pedagogy and curriculum depth. In doing so, we reject some popular but over‐simplistic applications of neuroscience to education. In sum, the education profession could benefit from embracing rather than ignoring cognitive neuroscience. Moreover, educationists should be actively contributing to the research agenda of future brain research.

Neural substrates of conditioned-cued relapse to drug-seeking behavior

Relapse to drug use following abstinence is a significant impediment in the long-term treatment of drug abuse and dependence. Conditioned stimuli are believed to be critically involved in activating drug craving and relapse to compulsive drug-taking behavior. Studies in humans and animal models have recently begun to identify the fundamental neural circuitry that mediates relapse following withdrawal from chronic drug self-administration. The current review summarizes key findings in this area that have converged on the amygdalar complex and regions of the frontal lobe as critical structures in conditioned-cued relapse. It is proposed that the amygdala is a key regulator of discrete stimulus–reinforcer associations, while the anterior cingulate and orbitofrontal cortex are critical regulators of relapse evoked by conditioned stimuli that predict drug availability. This corticolimbic circuitry may form the neural basis of multiple long-term conditioned associations produced by a variety of drugs of abuse ranging from psychostimulants to opiates. Future studies aimed at discerning the functional roles of these pathways will provide critical direction for the development of treatments for the prevention of relapse.

Programming for Maintenance: An Investigation of Delayed Intermittent Reinforcement and Common Stimuli to Create Indiscriminable Contingencies

Generalization across time or maintenance of behavior change is a fundamental concern for behavior analysts and educators that remains insufficiently understood. This study examined the maintenance of mathematics responding during and following delayed intermittent reinforcement when common stimuli were programmed across the treatment and maintenance phases. Two third-grade girls who were referred by their classroom teacher due to concerns in the area of mathematics participated. Students were exposed to baseline, contingent reinforcement, delayed intermittent reinforcement, and a maintenance condition. The maintenance condition followed exposure to delayed intermittent reinforcement and included common stimuli from the reinforcement condition, but did not include a contingency for correct responding. Both students exhibited substantial prolonged maintenance during this condition. Implications of these results for future research examining maintenance and applied programming for maintenance are discussed.

Cerebral Activation Associated with Sexual Arousal in Response to a Pornographic Clip: A 15O–H2O PET Study in Heterosexual Men

This study attempted to use PET and 15O–H2O to measure changes in regional cerebral blood flow (rCBF) during sexual arousal evoked in 10 young heterosexual males while they watched a pornographic video clip, featuring heterosexual intercourse. This condition was compared with other mental setups evoked by noisy, nature, and talkshow audiovisual clips. Immediately after each clip, the participants answered three questions pertaining to what extent they thought about sex, felt aroused, and sensed an erection. They scored their answers using a 1 to 10 scale. SPM was used for data analysis. Sexual arousal was mainly associated with activation of bilateral, predominantly right, inferoposterior extrastriate cortices, of the right inferolateral prefrontal cortex and of the midbrain. The significance of those findings is discussed in the light of current theories concerning selective attention, “mind reading” and mirroring, reinforcement of pleasurable stimuli, and penile erection.

Consciousness, higher-order thought, and stimulus reinforcement

Rolls defends a higher-order thought theory of phenomenal consciousness, mapping the distinction between conscious and non-conscious states onto a distinction between two types of action and corresponding neural pathways. Only one type of action involves higher-order thought and consequently consciousness. This account of consciousness has implausible consequences for the nature of stimulus-reinforcement learning.

PAVLOVIAN CONTINGENCIES AND RESISTANCE TO CHANGE IN A MULTIPLE SCHEDULE

According to theoretical accounts of behavioral momentum, the Pavlovian stimulus—reinforcer contingency determines resistance to change. To assess this prediction, 8 pigeons were exposed to an unsignaled delay‐of‐reinforcement schedule (a tandem variable‐interval fixed‐time schedule), a signaled delay‐of‐reinforcement schedule (a chain variable‐interval fixed‐time schedule), and an immediate, zero‐delay schedule of reinforcement in a three‐component multiple schedule. The unsignaled delay and signaled delay schedules employed equal fixed‐time delays, with the only difference being a stimulus change in the signaled delay schedule. Overall rates of reinforcement were equated for the three schedules. The Pavlovian contingency was identical for the unsignaled and immediate schedules, and response—reinforcer contiguity was degraded for the unsignaled schedule. Results from two disruption procedures (prefeeding subjects prior to experimental sessions and adding a variable‐time schedule to timeout periods separating baseline components) demonstrated that response—reinforcer contiguity does play a role in determining resistance to change. The results from the extinction manipulation were not as clear. Responding in the unsignaled delay component was consistently less resistant to change than was responding in both the immediate and presignaled segments of the signaled delay components, contrary to the view that Pavlovian contingencies determine resistance to change. Probe tests further supported the resistance‐to‐change results, indicating consistency between resistance to change and preference, both of which are putative measures of response strength.

Striatal dopamine in attentional learning: A computational model

This work describes a neural network model which simulates a discrete part of the dopaminergic striatal circuitry involved in reinforcement learning. We consider the proposal that learning by reinforcement is acquired by a heterosynaptic mechanism affecting plasticity of corticostriatal synapses, under the modulatory control of DA neurons. The simulation results seem to be successful since they are consistent with the literature that shows a transfer of DA neuron firing from the primary reinforcement stimulus to the conditioned stimulus after learning. We also present some simulations, using the same model, of two learning paradigms, blocking and overshadowing, which may be mediated by this dopamine circuitry.

EFFECTS OF UNSIGNALED DELAY OF REINFORCEMENT ON PREFERENCE AND RESISTANCE TO CHANGE

In Phase 1, pigeons were trained on a concurrent chain in which a 3‐s unsignaled delay of reinforcement was imposed on responding in a terminal link in some conditions. Preference for that terminal link was always reduced in comparison with conditions in which there was no delay, substantially so for 3 of the 4 pigeons. In Phase 2, pigeons responded in a two‐component multiple schedule. The scheduled rates of reinforcement were equal, but a 3‐s unsignaled delay was imposed in one component. Resistance of responding to prefeeding and extinction was reduced in the delay component for the same 3 subjects for which the data had shown strong effects of delay on preference. Systematic observation revealed differences in response topography. In the delay component, subjects oriented more closely to the key and responses were less forceful compared with the no‐delay component. Our results give further evidence that preference and resistance to change covary within subjects. However, they challenge the premise that the critical determiners of preference (i.e., terminal‐link value) and resistance to change (behavioral mass) may be quantified purely in terms of stimulus—reinforcer relations.

Gender, Mass Media and Social Change

Mass media plays an effective role in the present society. It informs, instructs, entertains, and performs an important role towards ushering social change. The role of mass media can be studied: as an agent of social change; as a reflector of dominant values; and as a rein forcer for strengthening dominant values.In the Indian context, media generally reflects and reinforces the dominant values. A study of TV commercials substantiates this. Advertising in the contemporary period has emerged as a powerful means. By indirectly projecting the social norms regarding the expected behaviour of women, TV commercials have acted as effective agents of reproducing patriarchal values. Patriarchal values are over-emphasized through ‘glamourization’ and ‘naturalization’ of domestic roles of women; glorification of the essential image of mother, representation of women in public life in soft roles and subordinate jobs, popularizing the image of women as sex objects; objects of beauty etc. A recent spot survey of TV commercials corroborates the above observations.Recognizing the potential of TV commericals in social life, it is imperative to initiate the process of social change through it for which some directions are suggested in the paper.

Combining CSs associated with the same or different USs.

In three experiments, hungry rats received appetitive training with four stimuli, A, B, X, and Y. In each Experiment, A and B were paired with one unconditioned stimulus (US; e.g. food pellets) whereas X and Y were paired with a second US (e.g. sucrose). Subsequently, rats responded more vigorously to combinations of stimuli associated with different USs (A-Y & X-B) than to combinations of stimuli associated with the same US (A-B & X-Y; Experiments 1, 2, & 3). This effect was observed when the stimuli were presented simultaneously and during the second elements of serial compounds (Experiments 2 & 3). Moreover, combining CSs associated with different USs resulted in a more marked CR than combining CSs that had each been paired with both US1 and US2 (Experiment 3). These results suggest that the sensory properties of appetitive reinforcers have an important influence on performance.

TRANSFER TESTS OF STIMULUS VALUE IN CONCURRENT CHAINS

We report two experiments that use transfer tests to investigate whether in concurrent chains the value of a terminal‐link stimulus is affected by the alternate terminal link. In Experiment 1, two groups of pigeons were trained on multiple concurrent‐chains schedules in which switching between the schedules was via pecking a changeover key. For one group, the terminal links were fixed‐interval 8 s versus fixed‐interval 16 s in one component and fixed‐interval 16 s versus fixed‐interval 32 s in the other component. For a second group, the terminal links were variable‐interval 10 s versus variable‐interval 20 s in one component and variable‐interval 20 s versus variable‐interval 40 s in the other. After sufficient baseline training had been given so that performances had stabilized, transfer tests were conducted in which the two chains with equal terminal‐link schedules were presented together as a new concurrent pair. For 6 of the 7 subjects, initial‐link responding changed fairly rapidly during the test in the manner predicted if the values of the terminal links were equal. In Experiment 2, pigeons were trained on multiple concurrent chains using a two‐key procedure, and the terminal links were the same variable‐interval schedules as in Experiment 1. After baseline training, transfer tests were conducted that assessed (a) the relative reinforcing strength of the terminal‐link stimuli in a novel initial‐link situation and (b) the relative ability of those stimuli to evoke responding. The data from the reinforcing strength test were consistent with those from Experiment 1, but those from the evocation strength test were not. Although this discrepancy shows that responding in transfer tests is not solely a function of stimulus value, the results from both experiments suggest, overall, that value is determined by the stimulus—reinforcer relation independently of the alternative terminal link.

Amphetamine-Induced Disruption of Latent Inhibition Is Not Reinforcer-Mediated

Latent inhibition (LI) refers to retarded conditioning to a stimulus that had been repeatedly preexposed without consequences, as compared with a nonpreexposed stimulus. Amphetamine disrupts LI, and this effect was suggested to result from enhanced switching to respond according to the stimulus–reinforcer contingency. Recently, it has been argued that amphetamine disrupts LI by increasing the impact of the reinforcer. This implies that amphetamine should produce stronger conditioning in the nonpreexposed group, and that its influence on LI can be modified only by changing reinforcer parameters. We report two studies, using an off-baseline conditioned emotional response procedure in rats licking for water, that question both predictions. In the first study, a meta-analysis based on 23 replications of the effect of amphetamine on LI, using tone as the preexposed stimulus, showed that LI is significantly attenuated due to drug-induced increased suppression in the preexposed groups only. The second study included two experiments, each using two shock intensities but different preexposed stimuli. Amphetamine disrupted LI at both shock intensities when the stimulus was a steady light, but this effect disappeared when the stimulus was three flashing lights. Thus, the effect of amphetamine could not be modified by manipulating shock intensity, but was modifiable by manipulating the nature of the preexposed stimulus. The results are inconsistent with the hypothesis that amphetamine-induced disruption of LI is solely mediated by drug-induced changes in the effects of reinforcers.