Stimulus-outcome Associations Research Articles

On Framing Effects in Decision Making: Linking Lateral versus Medial Orbitofrontal Cortex Activation to Choice Outcome Processing

Two correlates of outcome processing in the orbitofrontal cortex (OFC) have been proposed in the literature: One hypothesis suggests that the lateral/medial division relates to representation of outcome valence (negative vs. positive), and the other suggests that the medial OFC maintains steady stimulus-outcome associations, whereas the lateral OFC represents changing (unsteady) outcomes to prepare for response shifts. These two hypotheses were contrasted by comparing the original with the inverted version of the Iowa Gambling Task in an event-related functional magnetic resonance imaging experiment. Results showed (1) that (caudo) lateral OFC was indeed sensitive to the steadiness of the outcomes and not merely to outcome valence and (2) that the original and the inverted tasks, although both designed to measure sensitivity for future outcomes, were not equivalent as they enacted different behaviors and brain activation patterns. Results are interpreted in terms of Kahneman and Tversky's prospect theory suggesting that cognitions and decisions are biased differentially when probabilistic future rewards are weighed against consistent punishments relative to the opposite scenario [Kahneman, D., & Tversky, A. Choices, values, and frames. American Psychologist, 39, 341-350, 1984]. Specialized processing of unsteady rewards (involving caudolateral OFC) may have developed during evolution in support of goal-related thinking, prospective planning, and problem solving.

Interference and Time: A Brief Review and an Integration

Contemporary theories of associative learning have been preoccupied with the phenomenon of stimulus competition (attenuated responding to a target stimulus-outcome association if another stimulus trained together with the target is a better or more reliable predictor of the outcome). In recent years, reports of associative interference between associations trained separately have challenged associative learning theories, which provide no mechanism to account for this type of interference. Moreover, the ever-growing reports of temporal relationships between stimuli being a variable that determines the occurrence of stimulus competition and associative interference call for a reformulation of associative theory that can account for both interference and temporal learning effects. Here, we briefly review some of the central findings in stimulus competition, associative interference, and temporal learning, as well as a recently-proposed integration of these three seemingly disparate families of phenomena.

Summation and overexpectation with qualitatively different outcomes

In five experiments, a Pavlovian appetitive conditioning preparation was used with rats to explore the interaction of associations with different, but equivalently valued, outcomes. Experiment 1 demonstrated summation in magazine responding when two stimuli previously associated with different outcomes were combined. Experiments 2A and 2B found that pairing that stimulus compound with one of the outcomes led to a decrease in performance (overexpectation) for both of the stimulus elements. Experiments 3A and 3B confirmed that result but demonstrated the continued presence of the original stimulus-outcome associations. The results are consonant with a view in which replacing one outcome with another leads to an associative structure containing both stimulus-outcome associations and an outcome-independent depressive process.

Personality Disorders, Psychopathy, and Crime in a Norwegian Prison Population

Psychopathic individuals typically present with associative learning impairments under explicit learning conditions. The present study aimed to investigate whether the formation of stimulus-outcome associations, as well as updating of these associations after changed contingencies, could be improved by using rewards with sufficiently high subjective values. To this end, 20 psychopathic offenders, 17 non-psychopathic offenders and 18 healthy controls performed a passive avoidance task with a reversal phase under three motivational conditions, using naturalistic rewards. The subjective values of the rewards were assessed a priori for each individual participant using a visual analogue scale. The correspondence of these values to their internal representation was confirmed by analyses of brain potentials. Analyses using both signal detection theory and classical approaches indicated that psychopathic offenders performed worse compared to the other groups during passive avoidance learning. However, using the signal detection approach, we found this deficiency to be present only when a hypothetical reward was used (‘neutral reward’ condition), whereas psychopathic offenders performed similar to the other groups when naturalistic rewards could be obtained (‘low reward’ and ‘high reward’ conditions). Furthermore, traditional analyses suggested that psychopathic offenders had more hits than the other groups during reversal learning, but the signal detection approach indicated that no effects of group or condition were present. Analysis of win-stay and lose-shift behaviour showed that psychopathic offenders were less likely to stay with a rewarded response during passive avoidance learning in the neutral reward condition. In addition, regardless of experimental phase or condition, psychopathic offenders were less likely to stop responding to a particular stimulus after receiving negative feedback. Although the approaches employed did not lead to unequivocal results, our findings suggest that psychopathic offenders do have the ability to adapt their behaviour to environmental contingencies when positive reinforcers with sufficiently high subjective values are used.

Selective reinstatement of stimulus-outcome associations

In two experiments, the possibility of outcome-selective reinstatement of conditioned responding was examined. Evidence for outcome-selective reinstatement of previously extinguished appetitively conditioned magazine responses by rats was observed in both Pavlovian (Experiment 1) and discriminated instrumental conditioning (Experiment 2) procedures. In both experiments, stimulus-elicited magazine responses occurred more in the presence of a stimulus whose reinforcer was reinstated than they did in the presence of another stimulus whose reinforcer was not reinstated. This effect was observed after both brief and extensive amounts of extinction. Outcome-selective reinstatement of instrumental leverpressing, however, was not observed, although nonselective reinstatement of magazine responding and leverpressing was obtained in Experiment 2. Overall, the data from these studies challenge existing theories of reinstatement, and they provide additional evidence of the importance of outcome-specific processes in the control of learned performance.

Effects of several extinction treatments upon the integrity of Pavlovian stimulus-outcome associations

Pavlovian-to-instrumental transfer tests were used to assess the sensitivity of Pavlovian stimulus-outcome (S-O) associations to various extinction treatments in four appetitive conditioning experiments with rats. In Experiment 1, simple nonreinforcement of a stimulus was shown to have little impact on the ability of that stimulus to display outcome-specific transfer of control. Extinguishing a stimulus by pairing the stimulus with an alternative reinforcement in Experiment 2 also had no detectable effect on the S-O associations as assessed with the outcome-specific transfer measure. The third and fourth experiments, respectively, examined the impact of postconditioning exposures to random and explicitly unpaired S-O contingencies upon previously learned S-O associations. These treatments, as well, had no detectable harmful effects upon the integrity of the S-O associations. In contrast to the consistent failures of various extinction treatments to influence the ability of stimuli to display outcome-specific transfer, these treatments often did reduce the strength of conditioned responding initially trained to these stimuli. These results support the view that extinction entails the preservation of S-O associations as well as the parallel development of inhibitory stimulus-response associations. Other notions of extinction are also discussed.

An associative analysis of instrumental biconditional discrimination learning

Three different techniques were employed to analyze the associative structures mediating performance on an instrumental biconditional discrimination. In all three experiments, rats were trained concurrently on two tasks in which different stimuli signaled which one of two responses would be followed by reward. In each task, one response was rewarded in one stimulus and the other response was rewarded in the other stimulus. Correct responses earned pellets in one task and sucrose in the other task. The transfer procedure was used in Experiment 1A to identify whether or not an association developed between a biconditional discriminative stimulus and its instrumental outcome. Evidence was obtained that a biconditional cue elevated preferentially a new response trained with the same outcome. Experiments 1B and 3 examined the potential contribution of this stimulus-outcome association to biconditional performance by training the biconditional cues as signals (S-s) for the nonreinforcement of a different response. There was no evidence that this operation interfered with the ability of a biconditional cue to control performance of its correct response. In Experiments 1B and 2, the value of the instrumental outcome was reduced in an attempt to assess the contribution of stimulus-response associations to performance on the biconditional discrimination. The results of Experiments 1B and 2 reveal that correct responses were depressed following devaluation of the outcome used to train them, suggesting that learning about the response-outcome relation occurs. The implications of these results for binary and hierarchical models of instrumental learning are discussed.

Control of instrumental performance by Pavlovian and instrumental stimuli.

Two experiments used rats to examine the transfer of control of a stimulus to a new instrumental response. That transfer was successful to the degree that the stimulus and the response shared a common outcome. The transfer was more substantial, however, when the stimulus signaled the availability of that outcome for another instrumental response compared with signaling its occurrence in a Pavlovian manner. That result suggests that the stimulus-outcome associations formed during instrumental training are not reducible to a Pavlovian association.

Inhibitory associations between S and R in extinction

The associative changes that occur in extinction were investigated in four instrumental learning experiments. Experiment 1 used transfer based on a shared outcome to detect the continued presence of response-outcome (R-O) and stimulus-outcome (S-O) associations after a response had been nonreinforced in the presence of its controlling stimulus. Experiments 2–4 found that extinction resulted in the learning not to make a particular response in the presence of a particular stimulus, despite those continued R-0 and S-0 associations. These results suggest that extinction may superimpose upon those original outcome associations an inhibitory S-R association.

Long-Term Memory of Pigeons for Stimulus-Outcome Associations Involving Multiple Stimuli, each Seen in a Single Brief Trial

Experiment 1 explored performance of pigeons in two versions of a shortterm recognition memory procedure. In one version responding to entirely novel slides was rewarded, and responding to familiar slides (slides seen once, for 10 sec) was not rewarded; in the other version, responding to familiar slides was rewarded. Performance was initially below chance in both versions of the procedure. This result indicated that in this procedure associations were formed between the slides and the outcome (reward or non-reward) that followed their presentation. The result also suggested that the true capacity of pigeon recognition memory cannot be assessed using these procedures, as performance is inevitably disrupted by the bird's associative memory. The tendency of pigeons to form one-trial associations was exploited in Experiment 2. Phase 1 consisted of 16 two-session cycles: in Session 1 of each cycle, birds were shown 20 novel slides and were rewarded for responding to 10 of those slides; in Session 2, the same slides were shown again, with the same reinforcement contingencies. The birds showed significant overnight retention of the one-trial associations formed in Session 1 of each cycle. Phase 2 showed significant retention over periods of more than 20 days of associations involving 320 slides seen twice only. Phase 3 re-exposed for nine daily sessions one of the sets of 20 slides used in Phases 1 and 2; a high level of discrimination emerged rapidly and 4 (of 8) birds showed, by the end of training, no overlap in response rates to positive and negative slides. Comparative implications of the results are discussed.

Associations between an instrumental discriminative stimulus and multiple outcomes.

In 5 experiments, rats learned initial stimulus-outcome (S-O) associations in an instrumental training task. Experiment 1 and 2 examined the effects on those S-O associations of subsequent or concurrent training of the stimulis with another outcome. Experiment 3-5 examined the effects of subsequent nonreinforcement of the stimulus. In each case, the S-O association was assessed by its successful transfer to a different response that had earned the same outcome

Associations between an instrumental discriminative stimulus and multiple outcomes.

In 5 experiments, rats learned initial stimulus-outcome (S-O) associations in an instrumental training task. Experiment 1 and 2 examined the effects on those S-O associations of subsequent or concurrent training of the stimulis with another outcome. Experiment 3-5 examined the effects of subsequent nonreinforcement of the stimulus. In each case, the S-O association was assessed by its successful transfer to a different response that had earned the same outcome