Associative Learning Theory Research Articles

Separate amygdala subregions signal surprise and predictiveness during associative fear learning in humans

It has recently been suggested that learning signals in the amygdala might be best characterized by attentional theories of associative learning [such as Pearce-Hall (PH)] and more recent hybrid variants that combine Rescorla-Wagner and PH learning models. In these models, unsigned prediction errors (PEs) determine the associability of a cue, which is used in turn to control learning of outcome expectations dynamically and reflects a function of the reliability of prior outcome predictions. Here, we employed an aversive Pavlovian reversal-learning task to investigate computational signals derived from such a hybrid model. Unlike previous accounts, our paradigm allowed for the separate assessment of associability at the time of cue presentation and PEs at the time of outcome. We combined this approach with high-resolution functional magnetic resonance imaging to understand how different subregions of the human amygdala contribute to associative learning. Signal changes in the corticomedial amygdala and in the midbrain represented unsigned PEs at the time of outcome showing increased responses irrespective of whether a shock was unexpectedly administered or omitted. In contrast, activity in basolateral amygdala regions correlated negatively with associability at the time of cue presentation. Thus, whereas the corticomedial amygdala and the midbrain reflected immediate surprise, the basolateral amygdala represented predictiveness and displayed increased responses when outcome predictions became more reliable. These results extend previous findings on PH-like mechanisms in the amygdala and provide unique insights into human amygdala circuits during associative learning.

Repetition counts: repeated exposure increases intake of a novel vegetable in UK pre-school children compared to flavour–flavour and flavour–nutrient learning

Children are not consuming sufficient amounts of fruits and vegetables in their habitual diet. Methods derived from associative learning theories could be effective at promoting vegetable intake in pre-school children. The objective of the present study was to compare the effectiveness of different learning strategies in promoting the intake of a novel vegetable. Children aged between 9 and 38 months were recruited from UK nurseries. The children (n 72) were randomly assigned to one of three conditions (repeated exposure, flavour-flavour learning or flavour-nutrient learning). Each child was offered ten exposures to their respective version of a novel vegetable (artichoke). Pre- and post-intervention measures of artichoke purée and carrot purée (control vegetable) intake were taken. At pre-intervention, carrot intake was significantly higher than artichoke intake (P<0·05). Intake of both vegetables increased over time (P<0·001); however, when changes in intake were investigated, artichoke intake increased significantly more than carrot intake (P<0·001). Artichoke intake increased to the same extent in all three conditions, and this effect was persistent up to 5 weeks post-intervention. Five exposures were sufficient to increase intake compared to the first exposure (P<0·001). Repeated exposure to three variants of a novel vegetable was sufficient to increase intake of this vegetable, regardless of the addition of a familiar taste or energy. Repetition is therefore a critical factor for promoting novel vegetable intake in pre-school children.

Elemental representation and configural mappings: Combining elemental and configural theories of associative learning

In this article, we present our first attempt at combining an elemental theory designed to model representation development in an associative system (based on McLaren, Kaye, & Mackintosh, 1989) with a configural theory that models associative learning and memory (McLaren, 1993). After considering the possible advantages of such a combination (and some possible pitfalls), we offer a hybrid model that allows both components to produce the phenomena that they are capable of without introducing unwanted interactions. We then successfully apply the model to a range of phenomena, including latent inhibition, perceptual learning, the Espinet effect, and first- and second-order retrospective revaluation. In some cases, we present new data for comparison with our model's predictions. In all cases, the model replicates the pattern observed in our experimental results. We conclude that this line of development is a promising one for arriving at general theories of associative learning and memory.

A configural theory of attention and associative learning

A formal account of the relationship between attention and associative learning is presented within the framework of a configural theory of discrimination learning. The account is based on a connectionist network in which the entire pattern of stimulation presented on a trial activates a configural unit that then enters into an association with the trial outcome. Attention is assumed to have two roles within this network. First, the salience of the stimuli at the input to the network can be increased if they are relevant to the occurrence of reinforcement and decreased if they are irrelevant. Second, the associability of configural units can increase on trials when the outcome is surprising and decrease when the outcome is not surprising.

Conditioned suppression is an inverted-U function of footshock intensity

Most theories of associative learning assert that conditioned responding to a target cue is a monotonically increasing function of unconditioned-stimulus (US) intensity. In a lick suppression preparation with rats, a cue was paired with a 0.4-, 0.6-, 0.8-, 1.0-, 1.2-, or 1.4-mA footshock in Experiment 1a, and with a 0.3-, 0.8-, 1.3-, or 1.8-mA footshock in Experiment 1b. Subsequent suppression in response to the cue was an inverted-U function of the US intensity. In Experiment 2, we demonstrated that massive extinction of the training context and compound conditioning can each attenuate the response decrement caused by training with a high-intensity US. The sometimes-competing-retrieval model (Stout & Miller, Psychological Review 114:759-783, 2007) provides a better fit to these data than do several other models of associative learning.

Equilibria of perceptrons for simple contingency problems.

The contingency between cues and outcomes is fundamentally important to theories of causal reasoning and to theories of associative learning. Researchers have computed the equilibria of Rescorla-Wagner models for a variety of contingency problems, and have used these equilibria to identify situations in which the Rescorla-Wagner model is consistent, or inconsistent, with normative models of contingency. Mathematical analyses that directly compare artificial neural networks to contingency theory have not been performed, because of the assumed equivalence between the Rescorla-Wagner learning rule and the delta rule training of artificial neural networks. However, recent results indicate that this equivalence is not as straightforward as typically assumed, suggesting a strong need for mathematical accounts of how networks deal with contingency problems. One such analysis is presented here, where it is proven that the structure of the equilibrium for a simple network trained on a basic contingency problem is quite different from the structure of the equilibrium for a Rescorla-Wagner model faced with the same problem. However, these structural differences lead to functionally equivalent behavior. The implications of this result for the relationships between associative learning, contingency theory, and connectionism are discussed.

The Neuroscience of Learning: Beyond the Hebbian Synapse

From the traditional perspective of associative learning theory, the hypothesis linking modifications of synaptic transmission to learning and memory is plausible. It is less so from an information-processing perspective, in which learning is mediated by computations that make implicit commitments to physical and mathematical principles governing the domains where domain-specific cognitive mechanisms operate. We compare the properties of associative learning and memory to the properties of long-term potentiation, concluding that the properties of the latter do not explain the fundamental properties of the former. We briefly review the neuroscience of reinforcement learning, emphasizing the representational implications of the neuroscientific findings. We then review more extensively findings that confirm the existence of complex computations in three information-processing domains: probabilistic inference, the representation of uncertainty, and the representation of space. We argue for a change in the conceptual framework within which neuroscientists approach the study of learning mechanisms in the brain.

The role of outcome inhibition in interference between outcomes: A contingency‐learning analogue of retrieval‐induced forgetting

Current associative theories of contingency learning assume that inhibitory learning plays a part in the interference between outcomes. However, it is unclear whether this inhibitory learning results in the inhibition of the outcome representation or whether it simply counteracts previous excitatory learning so that the outcome representation is neither activated nor inhibited. Additionally, these models tend to conceptualize inhibition as a relatively transient and cue-dependent state. However, research on retrieval-induced forgetting suggests that the inhibition of representations is a real process that can be relatively independent of the retrieval cue used to access the inhibited information. Consistent with this alternative view, we found that interference between outcomes reduces the retrievability of the target outcome even when the outcome is associated with a novel (non-inhibitory) cue. This result has important theoretical implications for associative models of interference and shows that the empirical facts and theories developed in studies of retrieval-induced forgetting might be relevant in contingency learning and vice versa.

Feedback-related Brain Potential Activity Complies with Basic Assumptions of Associative Learning Theory

Feedback-related negativity (FRN) is an ERP component that distinguishes positive from negative feedback. FRN has been hypothesized to be the product of an error signal that may be used to adjust future behavior. In addition, associative learning models assume that the trial-to-trial learning of cue-outcome mappings involves the minimization of an error term. This study evaluated whether FRN is a possible electrophysiological correlate of this error term in a predictive learning task where human subjects were asked to learn different cue-outcome relationships. Specifically, we evaluated the sensitivity of the FRN to the course of learning when different stimuli interact or compete to become a predictor of certain outcomes. Importantly, some of these cues were blocked by more informative or predictive cues (i.e., the blocking effect). Interestingly, the present results show that both learning and blocking affect the amplitude of the FRN component. Furthermore, independent analyses of positive and negative feedback event-related signals showed that the learning effect was restricted to the ERP component elicited by positive feedback. The blocking test showed differences in the FRN magnitude between a predictive and a blocked cue. Overall, the present results show that ERPs that are related to feedback processing correspond to the main predictions of associative learning models.

連合学習理論は擬鼠主義の産物か

Many researchers have been interested in cognitive abilities of animals and investigated those issues by using several methods. Associative learning theory, which is mainly adopted in classical and operant conditioning literature, has been considered “simple” and sometime “appropriate” account of animal behavior from the point of view of Morgan's canon. In present article, criticism toward associative account and another reason why associative account is appropriate for animal behavior were reviewed, proposing proper usage of associative learning theory in complex animal behavior. One possible answer in present discussion is that associative learning theory is some kind of “intermediate language”, which can work as the bridge among neural mechanism, psychological faculty in animals, and anthropomorphic explanations.

Dissociating contingency awareness and conditioned attitudes: Evidence of contingency-unaware evaluative conditioning.

Whether human evaluative conditioning can occur without contingency awareness has been the subject of an intense and ongoing debate for decades, troubled by a wide array of methodological difficulties. Following recent methodological innovations, the available evidence currently points to the conclusion that evaluative conditioning effects do not occur without contingency awareness. In a simulation, we demonstrate, however, that these innovations are strongly biased toward the conclusion that evaluative conditioning requires contingency awareness, confounding the measurement of contingency memory with conditioned attitudes. We adopt a process-dissociation procedure to separate the memory and attitude components. In 4 studies, the attitude parameter is validated using existing attitudes and applied to probe for contingency-unaware evaluative conditioning. A fifth experiment incorporates a time-delay manipulation confirming the dissociability of the attitude and memory components. The results indicate that evaluative conditioning can produce attitudes without conscious awareness of the contingencies. Implications for theories of evaluative conditioning and associative learning are discussed.

Investigating cue competition in contextual cuing of visual search.

A fundamental principle of learning is that predictive cues or signals compete with each other to gain control over behavior. Associative and propositional reasoning theories of learning provide radically different accounts of cue competition. Propositional accounts predict that under conditions that do not afford or warrant the use of higher order reasoning processes, cue competition should not be observed. We tested this prediction in 2 contextual cuing experiments, using a visual search task in which patterns of distractor elements predict the location of a target object. Blocking designs were used in which 2 sets of predictive distractors were trained in compound, with 1 set trained independently. There was no evidence of cue competition in either experiment. In fact, in Experiment 2, we found evidence for augmentation of learning. The findings are contrasted with the predictions of an error-driven associative model of contextual cuing (Brady & Chun, 2007).

The Design Principles for Flow Experience in Educational Games

Educational games have to be well designed to incorporate learner engagement, an integral component of educational effectiveness. One foundation of designing educational engagement is flow theory. This article presents a flow framework that describes the building blocks of flow experience that can be used to design appealing and effective educational games for formal and informal learning contexts. The framework provides the principles for good educational game design, based upon associative, cognitive and situative learning theories, including engagement and pedagogic elements with a focus upon feedback and flow principles. Furthermore, the paper clarifies the relation between the flow experience and immersion. We tested the flow framework in the RealGame case study, which revealed that the RealGame business simulation game was well designed and effective at engaging students. We found that the university students’ flow experience in the game was high and the findings indicated that sense of control, clear goals and challenge-skill dimensions of flow scored the highest, but a rewarding experience and feedback dimensions also scored highly by the students. Overall, the results indicate that flow framework is a useful tool in studying game-based learning experiences.

Role of amygdala central nucleus in feature negative discriminations.

Consistent with a popular theory of associative learning, the Pearce-Hall (1980) model, the surprising omission of expected events enhances cue associability (the ease with which a cue may enter into new associations), across a wide variety of behavioral training procedures. Furthermore, previous experiments from this laboratory showed that these enhancements are absent in rats with impaired function of the amygdala central nucleus (CeA). A notable exception to these assertions is found in feature negative (FN) discrimination learning, in which a "target" stimulus is reinforced when it is presented alone but nonreinforced when it is presented in compound with another, "feature" stimulus. According to the Pearce-Hall model, reinforcer omission on compound trials should enhance the associability of the feature relative to control training conditions. However, prior experiments have shown no evidence that CeA lesions affect FN discrimination learning. Here we explored this apparent contradiction by evaluating the hypothesis that the surprising omission of an event confers enhanced associability on a cue only if that cue itself generates the disconfirmed prediction. Thus, in a FN discrimination, the surprising omission of the reinforcer on compound trials would enhance the associability of the target stimulus but not that of the feature. Our data confirmed this hypothesis and showed this enhancement to depend on intact CeA function, as in other procedures. The results are consistent with modern reformulations of both cue and reward processing theories that assign roles for both individual and aggregate error terms in associative learning.

An instance theory of associative learning

We present and test an instance model of associative learning. The model, Minerva-AL, treats associative learning as cued recall. Memory preserves the events of individual trials in separate traces. A probe presented to memory contacts all traces in parallel and retrieves a weighted sum of the traces, a structure called the echo. Learning of a cue-outcome relationship is measured by the cue's ability to retrieve a target outcome. The theory predicts a number of associative learning phenomena, including acquisition, extinction, reacquisition, conditioned inhibition, external inhibition, latent inhibition, discrimination, generalization, blocking, overshadowing, overexpectation, superconditioning, recovery from blocking, recovery from overshadowing, recovery from overexpectation, backward blocking, backward conditioned inhibition, and second-order retrospective revaluation. We argue that associative learning is consistent with an instance-based approach to learning and memory.

Adicción, impulsividad y curvas temporales de deseo

This study addresses, from theories of associative learning, the impulsivity and craving in patients with a diagnosis of substance dependence in abstinence, habitual use and relapse. We present a new method for the assessment of desire based on discount curve self description, in which patients describe how they perceive the temporal evolution and intensity of their desire. At the same time, through a brief questionnaire about emotion and desire -developed by the authors-and the Plutchik Impulsivity Scale, we set out to check whether positive or negative emotions increase, decrease, or are uninvolved in changes of desire and impulsivity. Perception of the evolution of desire in patients from the Relapse and Habitual Use groups is similar and presents a hyperbolic curve, but the Relapse group presented greater impulsivity. Moreover, a relationship is found in patients from the Relapse group between the hyperbolic increase of desire, the pleasurable states and impulsivity. The higher scores on impulsivity and the hyperbolic evolution of desire in patients who have recently suffered a relapse indicate the need to consider and specifically address these aspects as relevant to psychotherapy and/or pharmacological therapy.

Empirical support for an involvement of the mesostriatal dopamine system in human fear extinction.

Exposure therapy for anxiety disorders relies on the principle of confronting a patient with the triggers of his fears, allowing him to make the unexpected safety experience that his fears are unfounded and resulting in the extinction of fear responses. In the laboratory, fear extinction is modeled by repeatedly presenting a fear-conditioned stimulus (CS) in the absence of the aversive unconditioned stimulus (UCS) to which it had previously been associated. Classical associative learning theory considers extinction to be driven by an aversive prediction error signal that expresses the expectation violation when not receiving an expected UCS and establishes a prediction of CS non-occurrence. Insufficiencies of this account in explaining various extinction-related phenomena could be resolved by assuming that extinction is an opponent appetitive-like learning process that would be mediated by the mesostriatal dopamine (DA) system. In accordance with this idea, we find that a functional polymorphism in the DA transporter gene, DAT1, which is predominantly expressed in the striatum, significantly affects extinction learning rates. Carriers of the 9-repeat (9R) allele, thought to confer enhanced phasic DA release, had higher learning rates. Further, functional magnetic resonance imaging revealed stronger hemodynamic appetitive prediction error signals in the ventral striatum in 9R carriers. Our results provide a first hint that extinction learning might indeed be conceptualized as an appetitive-like learning process and suggest DA as a new candidate neurotransmitter for human fear extinction. They open up perspectives for neurobiological therapy augmentation.

Changes in attention to an irrelevant cue that accompanies a negative attending discrimination

Pigeons were trained in two experiments with negative patterning discriminations that were accompanied by an irrelevant cue. For Experiment1, the discriminations were of the form AX+ BX+ ABX-, where A and B were relevant, X was irrelevant, and + or - indicate whether or not reinforcement was delivered. The discriminations for Experiment2 were of the form A+ B+ AX+ BX+ ABX-. A subsequent test phase in both experiments revealed that the associability of A and B, and hence the attention paid to these stimuli, was less than the associability of X. The results are explained with a modified version of a configural theory of associative learning.

The learning of basic-level categories by pigeons: The prototype effect, attention, and effects of categorization

Pigeons were trained to classify composite faces of two categories created by mimicking the structure of basic-level categories, with each face consisting of an item-specific component and a common component diagnostic for its category. Classification accuracy increased as the proportion of common components increased, regardless of familiar and novel item-specific components, with the best discrimination occurring at untrained original faces used as the common components. A no-categorization control condition suggested that categorization gives rise to equivalence for item-specific components and distinctiveness for degrees of prototypicality. When some item-specific components were shared by exemplars of the contrasting categories, those that were not overlapping between the categories became the effective cues for the pigeons' responses. Implications of these results are discussed in the context of current categorization and associative learning theories.

Effects of dorsal or ventral medial prefrontal cortical lesions on five-choice serial reaction time performance in rats

Lesions of the rat medial prefrontal cortex (mPFC) produce behavioral impairments in the 5-choice serial reaction time (5CSRT) task, a widely used measure of sustained and selective visual attention. This experiment compared the effects of “dorsal” (centered on prelimbic and infralimbic cortices) and “ventral” (centered on dorsal peduncular cortex and tenia tecta) mPFC lesions on performance in a variant of the 5CSRT task. Because in some associative learning theories, the predictive validity of events determines the allocation of attention to them, we also examined the effects of cue validity in this task. Operant nosepoke responses to some briefly illuminated ports were consistently (100%) reinforced (CRF) with food, whereas for other ports, responding was reinforced on only 50% of the trials (partial reinforcement, PRF). Different patterns of impairment emerged depending on lesion location within the mPFC. Dorsal- and sham-lesioned rats responded more to CRF than to PRF cues, but ventral-lesioned rats responded similarly to CRF and PRF cues. Additionally, under some conditions of increased attentional demands, dorsal-lesioned rats failed to respond on many trials, whereas the impairment in ventral-lesioned rats was manifested as an increase in response errors. These results demonstrate separable roles for dorsal and ventral mPFC subregions in controlling attention.