Nonrewarded Trials Research Articles

Position Cues and Reward Memories as Compatible Components of Serial Learning

Rats were runway trained on three-trial series of different reward events, plain pellets (P), sucrose pellets (S), and nonreward (N). The series were constructed so that the position of the N event was always a valid cue for N. The relative validity of the P and S events, and their respective memories as cues, was manipulated by holding the reward kind constant, PSN or SPN (Experiment 1) and SNS and PNS, or SNP and PNP (Experiment 2), or by varying the reward kind from series to series (PSN, SSN, PPN, and SPN; Experiment 1). In both experiments, transfer tests to series with equivalent numbers of nonreward trials (NNN) or reward trials (SSS or PPP) showed that the pattern of approach, fast on P and S trials and slow on N, that developed under all training conditions was retained in the transfer tests. The transfer results implicate control by position cues under conditions that might otherwise have been interpreted only by reference to reward memories.

Extinction of Behavior in Infant Rats: Development of Functional Coupling Between Septal, Hippocampal, and Ventral Tegmental Regions

Learning of a behavior at a particular age during the postnatal period presumably occurs when the functional brain circuit mediating the behavior matures. The inability to express a learned behavior, such as inhibition, may be accounted for by the functional dissociation of brain regions comprising the circuit. In this study we tested this hypothesis by measuring brain metabolic activity, as revealed by fluorodeoxyglucose (FDG) autoradiography, during behavioral extinction in 12- and 17-d-old rat pups. Subjects were first trained on a straight alley runway task known as patterned single alternation (PSA), wherein reward and nonreward trials alternate successively. They were then injected with FDG and given 50 trials of continuous nonreward (i.e., extinction). Pups at postnatal day 12 (P12) demonstrated significantly slower extinction rates compared to their P17 counterparts, despite the fact that both reliably demonstrated the PSA effect, i.e., both age groups distinguished between reward and nonreward trials during acquisition. Covariance analysis revealed that the dentate gyrus, hippocampal fields CA1-3, subiculum, and lateral septal area were significantly correlated in P17 but not P12 pups. Significant correlations were also found between the lateral septal area, ventral tegmental area, and the medial septal nucleus in P17 pups. Similar correlative patterns were not found in P12 and P17 handled control animals. Taken together, these results suggest that septal, hippocampal, and mesencephalic regions may be functionally dissociated at P12, and the subsequent maturation of functional connectivity between these regions allows for the more rapid expression of behavioral inhibition during extinction at P17.

Ordinal position in the serial learning of rats

Rats were runway trained on each of two, three-trial series consisting of different varieties of reward (X, Y, and Z) and nonreward (N) serving as trial outcomes. The two series are represented as XNY and ZNN. Distinguishing the two series were different brightness and texture cues on the runway floor. Transfer tests, conducted after the rats had developed faster running for rewarded trials than for nonrewarded trials and slower running on Trial 2 of ZNN than on Trial 2 of XNY, provided evidence that trial position, rather than item memories, was controlling the discriminations. In Experiment 1, reversing the floor cues completely reversed the discriminations. In Experiment 2, transfer to NNN did not change the routine patterns of approach that had been established.

Extinction after regular and irregular reward schedules in the infant rat: Influence of age and training duration

Greater persistence in extinction is observed following inconsistent reward compared to that observed following consistent reward, an effect termed the partial reinforcement extinction effect (PREE). We report three experiments in which the extinction rates of random partially reinforced (PRF) or continuously reinforced (CRF) infant rat pups were compared to the extinction rate of pups trained with an alternative and regular schedule of partial reinforcement, known as patterned single alternation (PSA). In PSA, subjects learn to alternate speed of responding in anticipation of the regular alternation of reward and nonreward trials in the straight alley runway. In Experiment 1, 17-day-old PSA subjects showed CRF-like extinction rates; whereas in Experiment 2, in which extinction was initiated early in training prior to the onset of the PSA discrimination, PSA subjects showed prolonged, PRF-like extinction curves. In contrast, 12-day-old pups in Experiment 3 showed no reward-schedule-related differences in extinction, despite differences in behavior during acquisition. These results prompt a modification of Amsel's (1962) model of discrimination learning, and suggest the existence of a dissociation between different types of reward-related expectancies in the younger subjects.

The effects of parietal cortex lesions on an object/spatial location paired-associate task in rats

The present experiment was conducted in order to test the hypotheses (1) that the posterior parietal cortex (PPC) serves as a neural system that is critical for binding spatial location and object information in long-term memory and (2) that even restricted lesions of the PPC would result in similar deficits. Long-Evans rats were given either a large or a small PPC lesion or a control surgery under Nembutal anesthesia. After a 1-week recovery period, the rats were tested on either an object or a spatial location go/no-go successive discrimination task. After reaching criterion (a minimum of a 5 sec difference between reward and nonreward trials), they were trained on the other discrimination. After reaching criterion on the second discrimination, all of the rats were trained on a successive discrimination go/no-go task in which they had to remember which object/spatial location pairs had been associated with reward. As compared with controls, neither the small nor the large PPC lesion impaired object or spatial location discrimination. In the paired-associate object/spatial location task, both large and small PPC lesioned rats were impaired, relative to controls. These data suggest that the rodent PPC is not involved in object or spatial location discrimination but rather is involved in discrimination and long-term memory for the combination of object and spatial location information.

Social influences on morphine conditioned place preference

Social defeat in dominant male rats has previously been shown to result in a loss of rank, together with decreased consumption of a palatable sucrose solution, suggestive of a decrease in hedonic tone. The present study further investigated the effect of defeat on hedonic capacity using the conditioned place preference paradigm. Stable dominance hierarchies were determined in pairs of male PVG rats by repeated observation of agonistic behaviour at the onset of the dark phase of the dark-light cycle. Dominant animals and singly housed animals were then subjected to defeat by a male of the aggressive Tryon Maze Dull (TMD) strain, either 1 (singly housed animals only) 3 or 7 days prior to the first conditioning session. Each animal was tested for place conditioning with morphine (1 mg/kg i.p.) using one rewarded and one non-rewarded conditioning trial. In this paradigm, dominant animals as well as singly housed animals showed an increase in time spent in the drug-associated side but their submissive partners did not. Three days following the defeat of a dominant animal by a TMD, place conditioning to morphine was absent in the defeated animals, but was now present in their submissive partners. Seven days following defeat, conditioning was absent in defeated animals that were now submissive, but present in defeated animals that had maintained their dominant status. In isolated animals, place conditioning was absent 1 day following defeat, but was present 3 and 7 days post-defeat. These findings suggest that morphine-induced place conditioning is influenced by both the experience of defeat and the effects of defeat on social status.

Instrumental performance and time between reinforcements: Intimate relation to learning or memory retrieval?

According to scalar expectancy theory (SET), instrumental performance is determined by the ratio of the time between reinforcements in the trial (T T) to the overall time between reinforcements (T O). Groups for which theT O|T T ratio is the same should perform similarly. According to the sequential-memory view, the memory of nonreward becomes a signal for reward, and thereby promotes strong responding, when that memory is retrieved on a reward trial. In each of three runway investigations employing rats in a runway, two groups were compared that had the sameT O |T T ratio but that differed in the tendency to retrieve the memory of nonreward on a rewarded trial. In each investigation faster running on critical nonrewarded trials was associated with the group having the stronger tendency to retrieve the memory of nonreward on a rewarded trial. These findings are consistent with the predictions of the sequential-memory view, as well as with certain earlier findings, but are inconsistent with SET. It was indicated that the groups compared here were matched along a considerable number of dimensions—an unprecedented number for a varied reward investigation.

The sequential view: From rapidly fading stimulus traces to the organization of memory and the abstract concept of number

The development of the sequential approach to instrumental learning from about 1958 to the present is described. The sequential model began as an attempt to explain a particular class of neglected partial reward phenomena, those in which performance in acquisition and extinction is influenced by the particular sequence in which rewarded and nonrewarded trials occur in acquisition, and it was subsequently applied to a variety of other phenomena. Over time, the sequential model grew, sometimes through the replacement of older assumptions by novel ones, as when retrieved memories replaced stimulus traces, and sometimes simply through the addition of novel assumptions, such as that animals are capable of remembering retrospectively one, two, three or more prior nonrewarded outcomes-the N-length assumption. The most recent assumption added to the sequential model is that on a given trial the animal may utilize its memory of prior reward outcomes to anticipate both the current reward outcome and one or more subsequent reward outcomes. One way to view the sequential model is to say that it is a specific theory in various degrees of competition with other specific theories. Several examples of this are provided. Another way to view the sequential model, a more important way in my opinion, is to see it as a representative of a general theoretical approach, intertrial theory, which differs in fundamental respects from another much more generally utilized theoretical approach, intra-trial theory. I suggest that there is a substantial body of data that can be explained by inter-trial mechanisms but not by intratrial mechanisms. The future may well reveal that the inter-trial mechanisms have greater explanatory potential than the currently more popular intratrial mechanisms.

Acquisition and extinction of a shuttling response in honeybees (Apis mellifera) as a function of the probability of reward.

The effects of partial (intermittent) versus consistent reward on the acquisition and extinction of a shuttling response were studied in 3 experiments with foraging honeybees (Apis mellifera). Adding nonrewarded trials to rewarded trials (the equated-reinforcements design) improved performance in acquisition and increased resistance to extinction. Substituting nonrewarded trials for some rewarded trials (the equated-trials design), which had little effect on acquisition, also increased resistance to extinction but to a lesser extent than adding nonrewarded trials. Marked variations in the schedule of partial reward (the sequence of rewarded and nonrewarded trials) were without effect

An animal model of anhedonia: attenuation of sucrose consumption and place preference conditioning by chronic unpredictable mild stress

Chronic exposure to very mild unpredictable stress has previously been found to depress the consumption of, and preference for, highly palatable sweet solutions. The present study used the place conditioning procedure to investigate whether these effects result from a decreased sensitivity to reward. Rats were subjected to chronic mild unpredictable stress for a total of 4 weeks. During weeks 3 and 4, they received four training trials, in which rewards were presented in a distinctive environment, and four further non-rewarded trials in a different environment. The rewards used in different experiments were food pellets, dilute (0.7%) and concentrated (34%) sucrose solutions, and dl-amphetamine sulphate (0.5 and 1.0 mg/kg). In all experiments, non-stressed animals showed an increase in preference for the environment associated with reward; in stressed animals, these effects were abolished or greatly attenuated. Chronic unpredictable mild stress, which may be comparable in intensity to the difficulties people encounter in their daily lives, appears to cause a generalized decrease in sensitivity to rewards.

A test for S-S associations in a conditional counting task

Conditional numerical discriminations were established in runway-trained rats by presenting daily a pair of reward series that differed in the number of rewarded trials preceding a terminal nonrewarded trial. One kind of series contained three rewarded trials (RRRN or NRRRN), which always occurred in either a smooth, black runway or a rough, white runway (counterbalanced). The other series contained four rewarded trials (RRRRN or NRRRRN), which occurred in a runway of the other brightness and texture. The reward on each rewarded trial was 5.045-g Noyes pellets until Day 22. For Days 22-32, the series containing four Rs was altered so that the number of pellets on each respective trial was 14, 7, 3, and 1. To test whether S-S associations controlled the serial discriminations after Day 32, we rearranged the order of elements in the 14-7-3-1-0 series to 20-7-3-14-0. Rapid approach to the terminal series element on the test day would have implied the operation of S-S associations (14 pellets would have been a cue for upcoming 7, not 0), but the approach to that element remained slow, suggesting that the animals continued their counting of rewarded trials during the training with the S-S-relevant 14-7-3-1-0 series, irrespective of the order of the series elements.

The abolition of the partial reinforcement extinction effect (PREE) by amphetamine: Disruption of control by nonreinforcement

Two groups of rats were trained to run in a straight alley. The continuously reinforced (CRF) group received a food reward on every trial. The partially reinforced (PRF) group was rewarded on a quasi-random 50% schedule. d-Amphetamine 1 mg/kg was administeted to PRF animals in acquisition in a 2×2 design, i.e., drug-no drug on reinforced trials and drug-no drug on nonreinforced trials. In four CRF groups, the drug was administered in the same sequence as in the PRF groups. Following acquisition, all animals were given 4 days of CRF retraining and tested in extinction. No drug was given in retraining and extinction. The PREE, i.e., increased resistance to extinction exhibited by PRF animals as compared to CRF animals, was obtained in groups which received placebo on all acquisition trials or amphetamine on rewarded trials and placebo on nonrewarded trials. The PREE was abolished when amphetamine was administered throughout the acquisition trials or on nonrewarded trials, irrespective of drug treatment on rewarded trials.

Partial reinforcement delayed extinction effect after regularly alternating reward training

When extinction is delayed very long, the superior resistance to extinction of the random schedule group relative to the alternating schedule group disappears (partial reinforcement delayed extinction effect, PRDE). Two experiments assessed the effects of reinforcement/nonreinforcement on Trial 1 on the PRDE. Following extended partial reinforcement acquisition training in a runway, rats received extinction training after a short (1-day) or long (23-day) retention interval. The schedules used in Experiment 1 were: a single-alternation (SA) schedule beginning each day with a rewarded (r) trial, for Group r-SA; an SA schedule beginning with a nonrewarded (n) trial, for Group n-SA; and a random (Rd) schedule, for Group Rd. The schedules and group names used in Experiment 2 were r-SA, Rd, and r-Rd. The results were that (1) rats given r-SA schedules yielded considerable resistance under delayed extinction, (2) those given Rd and r-Rd schedules showed a decline in resistance to extinction over a long retention interval, (3) those given the n-SA schedule showed relatively low resistance at both retention intervals, although retention deficit was not greater than in the case of the Rd schedule, and thus, (4) the PRDE was found in both experiments, although only weakly in Experiment 1. The results indicated that a regularly alternating reward pattern was a more important determinant than was type of reward on Trial 1 for the PRDE. The PRDE due to differential retention deficits among schedules is discussed on the basis of dual-process associative sequential mechanisms and cognitive rule-encoding mechanisms.

Alleyway paper flooring as a locus of nondifferential components of reward and nonreward odors

The locus of discriminable differences in conspecific odors arising from rewarded (R) and nonrewarded (N) trials administered to rats in a straight alleyway was examined by collecting the paper flooring from the alleyway after a squad of donor rats had received R or N. Paper flooring was also collected after a different squad of donors, with no history of alleyway R and N, merely walked on the paper. Different test rats were trained to discriminate between one of these kinds of alleyway paper and clean paper, and then each test rat was successively shifted through two more discriminations involving each of the other kinds of alleyway paper and clean paper. Results indicate that all of the alleyway papers retained olfactory components, which were discriminable from clean paper, and that interchanging the kinds of alleyway paper produced little disruption of discrimination. Implications for the volatility and nondifferential nature of components captured on the paper are discussed.

Species-typical taxic behavior and event-reinforcer interactions in conditioning

The role of species-typical taxic behavior in conditioning was studied using thigmotaxis (wall hugging) in rats. Running toward or staying near walls in an open-field apparatus was assumed to be compatible with the rat's defensive behavior, and running to or staying in open spaces was assumed to be incompatible. With large food rewards no differences between the center and side responses were found in acquisition under continuous or intermittent reward, but the center response showed greater suppression after contingent punishment was applied on nonreward trials. Two additional experiments showed that differences between center and side responses occur during extinction when avoidance of shock but not when food is the reinforcer. The results suggest that instances of response-reinforcer interactions may be predicted by knowledge of species-typical defensive behavior and that reactions elicited by shock are not always identical to those elicited by frustration in their consequences for the performance of conditioned behavior.

The inhibitory effect of pretrial reinforcement on instrumental conditioning

It has been shown previously that pretrial feeding retards acquisition of an instrumental response. The present experiment tested three hypotheses for this effect: priming, overshadowing and conditioned inhibition. Rats were trained in a runway task with a partial schedule of food reinforcement. Rats that were prefed immediately prior to rewarded trials showed a decrement in acquisition when compared with subjects prefed 180 seconds pretrial. Other subjects prefed immediately prior to nonrewarded trials showed a similar decrement in acquisition, even though the pretrial and goal reinforcers occurred on separate trials. Thus, the adverse effects of prefeeding occurred in the absence of conditions necessary for priming or overshadowing to occur. The results suggest there may be an inhibitory aftereffect of food rewards which can become conditioned to subsequent stimuli.

Sparing of patterned alternation but not partial reinforcement effect after infant and adult hippocampal lesions in the rat.

The effect of hippocampal lesions was assessed in patterned (single) alternation (PA) and the partial reinforcement extinction effect (PREE), the two reward-schedule effects that appear earliest in ontogeny. Three groups of rat pups, with appropriate controls, were tested for each effect: lesion as infant/test as infant, lesion as infant/test as adult, and lesion as adult/test as adult. Hippocampal lesions had no effect on PA in any of the three conditions except for a suggestion that the effect was mildly attenuated in the animals given lesions as infants and tested as adults: The pups were able to discriminate rewarded from nonrewarded trials and to inhibit responding on nonrewarded trials. On the other hand, the PREE was eliminated under all conditions of testing, in each case because of an increase in persistence following continuously reinforced acquisition. The results are discussed in terms of the functional maturation of the hippocampus and a possible dissociation of mechanisms that mediate response suppression in PA and in the PREE in infant rats.

Effects of anxiolytic drugs on the partial reinforcement extinction effect in runway and Skinner box.

The partial reinforcement extinction effect (PREE) in the runway is reduced by anxiolytics in a non-state-dependent manner when a 24 h inter-trial interval is used, but there is some doubt as to the nature of the drug effects when shorter intervals are used. Experiment 1 repeated a study by Gray (1969), in which ambiguous results were obtained using eight trials/day. It demonstrated that the anxiolytic barbiturate, sodium amylobarbitone, given both in acquisition and extinction does not reduce the PREE. It confirmed Gray's observation that the PREE is abolished if the drug is given in acquisition but not in extinction. This suggests that a 24 h inter-trial interval is one critical factor in non-state-dependent reduction of the PREE. Experiments 2 and 3 tested the effects of the anxiolytic benzodiazepine, chlordiazepoxide, on the PREE with a 24 h inter-trial interval in the Skinner box. The basic task was a single FR5 sequence terminating in delivery of 10 (Experiment 2) or 20 (Experiment 3) reward pellets each day. There were 10 acquisition trials and partially reinforced rats received either three (Experiment 2) or four (Experiment 3) non-rewarded trials. The drug abolished the PREE in Experiment 2 and effectively reversed it in Experiment 3. These results confirm previous work with this drug in the runway (Feldon and Gray, 1981) and extend them to a very different experimental situation. These results support the idea that the PREE depends on different processes with different acquisition parameters; and that when a 24 h inter-trial interval is used the PREE is largely produced by some general process, probably the counterconditioning of conditioned frustration, which is sensitive to anxiolytic drugs. They also demonstrate very clearly the paradoxical effects of the anxiolytic drugs when given in both acquisition and extinction: they generally increase resistance to extinction in continuously reinforced animals, but block the increase in resistance (the PREE) produced by behavioural schedules.

Performance of goldfish (Carassius auratus) in patterned sequences of rewarded and nonrewarded trials.

The performance of goldfish was studied in a series of experiments with patterned sequences of reward (R) and nonreward (N) for response to a given stimulus (S3). Trials with two other, readily discriminable stimuli, response to one always rewarded and response to the second never rewarded, were interpolated in such a way that the outcome of any S3 trial could not be predicted from the events of the immediately preceding trial. The main purpose was to look for control of response to S3 based on memory of N like that found in previous work with pigeons. In Experiments 1 and 2, blocks of R and N trials with S3 were programmed, R trials in the first half of each session and N in the last half, or N trials in the first half and R in the last half. In Experiment 3, successive acquisition and extinction of response to S3 was studied, with the N-to-R and R-to-N transitions either within sessions or between sessions. In Experiment 4, the effects of partial and consistent reinforcement on extinction of response to S3 were compared. The results obtained are similar in important respects to those for pigeons but different in their failure to demonstrate control of performance by memory of N.

Aggression in Rats as a Function of Target Location in a Double Alley

After training in a double alley for 100% food reward in each of two goalboxes, 14 male Long-Evans hooded rats were run on a partial reinforcement schedule with a target rat present in either the first goalbox or the second runway. The incidence of attack was objectively recorded in both locations. Significantly higher levels of aggression were recorded during nonreward trials than during reward trials in the goalbox, or during either nonreward or reward trials in the runway. The results are interpreted as support for the hypothesis that, in general, aggression is elicited by frustration, but that under certain conditions alternative responses to frustration may preclude aggression.