Operant evaluative conditioning is sensitive to US-revaluation

A comparison of neuronal reactions during classical and instrumental conditioning under similar conditions

Chapter 15 - Comparison of Operant and Classical Conditioning of Feeding Behavior in Aplysia

MARKE. BOUTON Learning and Behavior: A Contemporary Synthesis Sinauer, 2007, 419 pages (ISBN: 9780878930630, $94.95, Harcover) Reviewed by DARLENE SKINNER In Learning and Behavior: A Contemporary Synthesis, Mark Bouton has created a student-friendly textbook for an undergraduate course in animal learning that is both comprehensive and current. Bouton presents the major theoretical issues and findings in their historical context, describing what life was like back then in terms of the thinking and technology. He uses various tools to facilitate students' understanding of the material covered. In addition to the end of chapter summaries and list of key terms, which are standard for undergraduate texts, Bouton summarizes the main points and issues at the end of each topic/section with headings such as What does it all mean? and How do they do it? On the first page of each chapter, Bouton also briefly summarizes the key points and issues of the previous chapter before moving on to the goals of the current chapter. The book is well illustrated with pictures of many of the researchers in the field, as well as diagrams and graphs showing important concepts and data. In the introductory chapter, Bouton describes what the field of animal learning is like today and how it got to be that way. He deals briefly with the philosophical and biological roots of learning theory before moving to the early learning theorists (Watson, Skinner, and Tolman). Here he lays out the basics of stimulus and response learning, the two forms of learning that will be dealt with in more detail in the rest of the book. He describes how these two forms of learning interact and can be used to examine any behavioural situation. He uses real-life examples of behaviour, from bubble blowing to drug-taking, to illustrate his points. He introduces common terms to describe both types of learning and by the end of this first chapter the student should be able to pick out the stimulus (S), response (R), and reinforcing stimulus (S*) from a variety of behavioural situations. In the second chapter, Learning and Adaptation, Bouton explains that the two types of learning described in Chapter 1 are so prevalent because they allow animals (including humans) to adapt to their environment. Pavlovian and operant conditioning follow similar rules because of this overlap in function. He goes on to describe several examples of adaptation in both operant and Pavlovian conditioning, including fear conditioning and taste aversion learning, that are referred to repeatedly throughout the remainder of the book. He then describes many of the parallels between the two types of learning, including extinction, timing effects, size effects, and preparedness. Right from the start, Bouton emphasizes how the two types of learning, often treated separately, are very much alike. Quite a lot of information is conveyed in the first two chapters about operant and Pavlovian conditioning but Bouton accomplishes this using a common vocabulary for the two types of learning and using everyday examples students will surely relate to in addition to the typical laboratory experiments. As is typical of undergraduate textbooks, from Chapter 3 on, Bouton treats Pavlovian and operant conditioning separately. Major topics in Pavlovian conditioning are covered in Chapter 3, including the basic conditioning experiment, the various methods for studying Pavlovian conditioning, latent inhibition, US pre-exposure, conditioned inhibition, contingency, blocking, and unblocking. …

Categories of Learning and the Problem of Definition: COMMENTS ON PROFESSOR GRANT'S PAPER

Previous studies showed that baroreflex control of heart rate is impaired during operant shock avoidance conditioning and classical aversive conditioning. However, the effects of such "emotionally stressful" paradigms on the ability of the baroreflex to control arterial pressure have not been directly assessed. We prepared the carotid sinus regions of dogs for reversible isolation from the systemic circulation, and we derived complete stimulus-response relations for the effects of carotid sinus pressure on both heart rate and arterial pressure. For any given carotid sinus pressure, arterial pressure and heart rate were higher during operant shock-avoidance conditioning and during classical aversive conditioning than in a neutral environment, which indicates an upward resetting of the baroreflex. However, threshold and saturation carotid sinus pressures were unaffected by operant conditioning or classical conditioning, which indicates that the baroreceptors themselves were not reset. The ranges over which the carotid baroreflex could vary arterial pressure and heart rate were significantly increased during both operant conditioning and classical conditioning. Baroreflex gain was unchanged during operant conditioning and was significantly increased during classical conditioning. We conclude that the baroreflex is not impaired during operant shock-avoidance conditioning or classical aversive conditioning in dogs. However, the baroreflex is reset and regulates blood pressure at an elevated level.

SummaryHow the brain uses success and failure to optimize future decisions is a long-standing question in neuroscience. One computational solution involves updating the values of context-action associations in proportion to a reward prediction error. Previous evidence suggests that such computations are expressed in the striatum and, as they are cognitively impenetrable, represent an unconscious learning mechanism. Here, we formally test this by studying instrumental conditioning in a situation where we masked contextual cues, such that they were not consciously perceived. Behavioral data showed that subjects nonetheless developed a significant propensity to choose cues associated with monetary rewards relative to punishments. Functional neuroimaging revealed that during conditioning cue values and prediction errors, generated from a computational model, both correlated with activity in ventral striatum. We conclude that, even without conscious processing of contextual cues, our brain can learn their reward value and use them to provide a bias on decision making.

This chapter argues that despite the questionable status of Miller’s data, there are sufficient suggestions of operant control of some autonomic functions to keep one from distinguishing between types of conditioning in this manner. It deals with what may be described as one aspect of a subcategory of a closely related topic. The “closely related topic” is the attempt to distinguish between Pavlovian and operant conditioning. In the mid-1960s, a number of researchers argued that autonomic responses could be conditioned by either operant or Pavlovian methods, therefore invalidating them as a technique for distinguishing between operant and Pavlovian conditioning. The particular response that occurs in a given Pavlovian conditioning situation is one that reflects the knowledge of the temporal relationship among the central representations of events. Some classification of responses that cross-cuts the skeletal-autonomic distinction may turn out to be more relevant to the distinction between operant and Pavlovian conditioning.

Objective information was obtained from 89 textbooks in the psychology of learning published from 1952 through 1995 to answer questions about their coverage of classical conditioning. We found (a) that classical conditioning consistently received less coverage than instrumental and operant conditioning, (b) that the distinction of classical and instrumental conditioning has become established as a textbook-construction device, (c) that use of the classical-instrumental distinction by textbook authors has "hardened" after a period of "fluidity" in the 1970s, (d) that the currency of references in the basic classical (as well as instrumental) conditioning chapters diminished from 1952 to 1995. Findings are discussed in regard to the possibility that, as a class, and in comparison to other textbooks, the textbooks of the psychology of learning fall short of fulfilling some of their recognized roles.

Objective information was obtained from 89 textbooks in the psychology of learning published from 1952 through 1995 to answer questions about their coverage of classical conditioning. We found (a) that classical conditioning consistently received less coverage than instrumental and operant conditioning, (b) that the distinction of classical and instrumental conditioning has become established as a textbook-construction device, (c) that use of the classical-instrumental distinction by textbook authors has "hardened" after a period of "fluidity" in the 1970s, (d) that the currency of references in the basic classical (as well as instrumental) conditioning chapters diminished from 1952 to 1995. Findings are discussed in regard to the possibility that, as a class, and in comparison to other textbooks, the textbooks of the psychology of learning fall short of fulfilling some of their recognized roles.

We show how a minimal components requirement and very low resource demanding field-programmable gate array (FPGA) implementation of an adapted version of the synapto-dendritic Kernel Adapting Neuron (SKAN) model can be used to underlie two of the most basic learning processes: classical conditioning (CC) and operant conditioning (OC). In the CC architecture, this adapted SKAN model is used in a spiking neural network (SNN) to implement spike-timing-dependant-plasticity (STDP). However, in order to achieve a functioning OC architecture, a new STDP-inspired learning process is introduced. The modified CC architecture, new OC architecture, adapted SKAN model and new STDP-inspired process represent the four contributions presented here, along with simulation results on a FPGA which shows their adequacy in supporting CC and OC learning behaviors.

Immunohistochemical assessment of mesotelencephalic dopamine activity during the acquisition and expression of Pavlovian versus instrumental behaviours

A robot is presented whose behavior is based on two fundamental types of learning in the animal world: Classical Conditioning (CC) and Operant Conditioning (OC). It is shown how both share Spike-Timing-Dependent-Plasticity (STDP) as learning process for a Spiking Neural Network (SNN). STDP was implemented on a Field-Programmable Gate Array (FPGA) with very low-demanding resources, using an adaptation of the Synapto-dendritic Kernel Adapting Neuron (SKAN) model. Moreover, it is shown how a 3-way version of STDP is needed to allow for OC. The robot was designed to use the CC and OC neuronal architectures proposed in this paper and was tested in a dynamic environment, which consisted of a maze with changing features. It was successful in presenting both types of learning. This paper thus validates an architecture with an important potential for very large scale time-dependent parallel data analysis, with high capacity of adaptation in a dynamic environment.

Biological Constraints on Instrumental and Classical Conditioning: Implications for General Process Theory

Chapter 8 presents the behavioral approaches of classical and operant conditioning. While classical conditioning is not widely used in corrections, except occasionally for special populations (e.g., people who commit sex offenses), operant conditioning, in the form of token economies, is pervasive, especially in correctional settings for juveniles. And, of course, later key components of social learning and cognitive–behavioral approaches grew out of classical and operant conditioning—the earliest behavioral models. The key concepts of behavioral interventions (techniques of effective reinforcement and appropriate punishment and disapproval) are so effective that many agencies are now training all correctional staff to use them. The use of technology through virtual reality techniques is also presented.

The cockroach, Leucophaea maderae, can be trained in an associative olfactory memory task by either classical or operant conditioning. When trained by classical conditioning, memory formation is regulated by a circadian clock, but once the memory is formed, it can be recalled at any circadian time. In contrast, when trained via operant conditioning, animals can learn the task at any circadian phase, but the ability to recall the long-term memory is tied to the phase of training. The optic lobes of the cockroach contain a circadian clock that drives circadian rhythms of locomotor activity, mating behavior, sensitivity of the compound eye to light, and the sensitivity of olfactory receptors in the antennae. To evaluate the role of the optic lobes in regulating learning and memory processes, the authors examined the effects of surgical ablation of the optic lobes on memory formation in classical conditioning and memory recall following operant conditioning. The effect of optic lobe ablation was to "rescue" the deficit in memory acquisition at a time the animals normally cannot learn and "rescue" the animal's ability to recall a memory formed by operant conditioning at a phase where memory was not normally expressed. The results suggested that the optic lobe pacemaker regulates these processes through inhibition at "inappropriate" times of day. As a pharmacological test of this hypothesis, the authors showed that injections of fipronil, an antagonist of GABA and glutamate-activated chloride channels, had the same effects as optic lobe ablation on memory formation and recall. The data suggest that the optic lobes contain the circadian clock(s) that regulate learning and memory processes via inhibition of neural processes in the brain.