Post-reinforcement Synchronization Research Articles

The magnitude of post-reinforcement EEG synchronization (PRS) in cats reflects learning ability

Electrocortical alpha-like activity, termed post-reinforcement synchronization (PRS), observed during operantly conditioned behavior, was quantified in 25 cats and plotted against the number of training sessions required for each to learn to press a lever for milk reward. A significant correlation ( r = 0.77 ; P < 0.001) was found between PRS magnitude and learning ability. The time required for learning correlated with the amount of perseveration in partially reinforced, i.e. ineffective, behavioral modes.

Pulvinar and lateral geniculate neuronal activity in the cat during operantly conditioned appetitive behavior

Cats were trained to press a lever for 0.5–1.0 ml of milk reward both in the presence and absence of ambient light. ‘Floating’ microelectrodes were implanted in the pulvinar and lateral geniculate nuclei to allow chronic recording from single or multiple neurons over a period of 1–16 days. Thirty-one of 36 pulvinar neurons (86%) showed a consistent periodicity in the firing rate that was clearly related to certain phases of operant behavior. In the presence of ambient light, the firing rate was maximal shortly prior to or immediately after a rewarded bar press, particularly during the animal's initial contact with reward (the first lap). This burst of activity was followed by an abrupt and strong inhibition of firing which was, in most instances, associated with high voltage slow wave (6–9 c/sec) postreinforcement synchronization (PRS) of the electroencephalogram over the visual and association cortices as well as in the pulvinar. Postreinforcement inhibition of neuronal activity was also observed in the lateral geniculate. Neuronal firing in both nuclei dropped significantly below the respective mean rate observed for each nucleus during relaxed wakefulness. In the dark, however, the reward-induced inhibition was abolished in both nuclei. Moreover, some pulvinar neurons in the dark showed a conspicuous and significant increase in firing during the consummatory response (dark reversal pattern). These findings suggest that a transient but powerful inhibitory process triggered by positive reinforcement depends on visual input. Firing patterns of some pulvinar neurons were noted to be entrained not only by the rhythm of PRS oscillations (inhibitory phasing and postinhibitory discharges) but also by the much slower 3–4 c/sec rhythm of lapping and licking. Furthermore, the patterns of entrainment by initial lapping appear to be different from those observed during the later phase of the consummatory response, thus indicating that the pulvinar may receive information regarding the content of the delivery cup. No entrainment of neuronal activity was observed in the lateral geniculate. These findings reveal a remarkable plasticity in the pulvinar neurons that is consistent with their role in polysensory integration.

Electrical activation of visual pathways substitutes for tonic light input in triggering EEG correlates of food reward during conditioned behavior in cats

Cats trained to press a lever for 1 cc of milk reward normally show during the consummatory response high voltage 6--8 c/sec EEG synchronization associated with epicortical positive steady potential shift over the primary and secondary visual projections. The emergence of this postreinforcement synchronization (PRS) and Reward Contingent Positive Variation (RCPV) is known to depend upon appropriate gustatory input and presence of ambient light, although visual perception of reward and/or environment is not essential as shown in cats wearing translucent "milky" contact lenses. Training the animals in a paradigm in which a "light-off" cue signaled the availability of reward, and thus assigned positively reinforcing quality to the dark condition, also failed to restore the PRS-RCPV phenomenon in the absence of light. However, brief electrical stimuli applied to either the optic tract or the lateral geniculate nucleus substituted for unpatterned light input and fully restored the PRS-RCPV in the dark. The suprathreshold stimuli were effective only during the consummatory response. Maximum effect could be produced during a brief time period between 0.8 and 1.5 sec after the onset of consummatory response as judged by lapping activity, thus showing the specificity of the effect of visual input. Even during a relaxed wakefulness after satiation or during slow wave sleep or REM sleep the same electric stimuli were ineffective although they produced well-developed evoked potentials with all characteristic wave components. The results indicate that brain pathways utilize unpatterned i.e. noisy visual input in complex integrative processes involving gustatory input.

Dependence of reward contingent positive variation (RCPV) and cortical synchronization on visual input in the cat

Cats trained to press a lever for milk reward display high amplitude low frequency electrocorticographic post-reinforcement synchronization (PRS) associated with epicortical steady potential shift, termed Reward Contingent Positive Variation (RCPV), over the parieto-occipital cortex. Previously, it was found that the PRS-RCPV phenomenon depends on unpatterned light input devoid of any conceivable information or conditional property. Although training in the dark most likely eliminated the “novelty” factor and the resulting orienting reaction to “light off” as a cause of PRS-RCPV suppression, a possibility could not be excluded that the absence of ligh had a negatively reinforcing property preventing the emergence of PRS-RCPV. Hence, in the present study an experimental paradigm was used in which “light off” and/or “tone on” cues signalled the availability of reward, the “light off” periods extending through the consummatory response. Seven out of 9 cats, despite 2–4 months of daily training, and 85% correct timing of bar press performance, never showed a PRS-RCPV response (one animal showed only sporadic and poorly developed patterns). However, when the light was turned on during consumption, in 85% of such tests a fully developed PRS-RCPV emerged in a stimulus-bound manner. One animal was an exception and showed PRS-RCPV both in the absence and presence of light. It was concluded that light input plays a crucial role in the emergence of the PRS-RCPV phenomenon and therefore also participates in the integration of gustatory input.

Behavioral and electrophysiological correlates during flash-frequency discrimination learning in monkeys

1. 1. An attempt has been made to determine the correlation between animals' behavior and the alteration of the electrical activities especially of the lateral geniculate nucleus (LGN) and the striate and parastriate cortex (OC and OA). Four monkeys ( Macaca mulatta) were trained to discriminate the frequency of flash trains (2/sec and 4/sec for 5 sec), and to pull the lever for 1 ml of juice reward. 2. 2. Computer-averaged evoked potentials (EPs) of the LGN, OC and OA induced by flash stimuli showed similar pattern and amplitude for each response, when the animal was not engaged in learning performance. 3. 3. During the elaborate performance of discriminatory learning the EPs induced by the initial several stimuli were augmented in the LGN, and reduced in the OC and OA. This dissociative alteration is discussed as relevant to the animals' increased alertness or selective attention to flash. 4. 4. The EPs of the LGN, OC and OA induced by the succeeding stimuli were markedly augmented, when the animals pulled the lever to be rewarded during the presentation of the positive conditioned stimuli. On the other hand, the EPs were returned to the control level in the case of the negative conditioned stimuli. 5. 5. The synchronized EEG of 6–9/sec was observed in the pulvinar nucleus, mesencephalic reticular formation and the frontal, parietal and infero-temporal cortex, when the animals were rewarded during the positive conditioned stimuli. 6. 6. It is suggested that the neural mechanisms for the post-reinforcement synchronization may be in the subcortical structures with the common networks responsible for the synchronized sleep.

Visual activity during post-reinforcement synchronization

Visual activity during post-reinforcement synchronization

Post-reinforcement EEG synchronization depends on the operant response

Cats who were deprived of food and water were trained to press a lever for milk reward on a variable-interval schedule. The electroencephalogram (EEG) from the primary visual area was recorded. All cats' EEGs eventually developed the PRS phenomenon, i.e., a dramatic shift from desynchronized to highly synchronized large amplitude waves after delivery and during ingestion of the milk reward. The lever was then taken out of the test chamber, and the milk reward was presented automatically at variable intervals. In this free-reward situation, the post-reinforcement synchronization (PRS) waves gradually disappeared from the EEGs. This was a stable condition, lasting over many trials. The test situation was then changed back to the one requiring lever-pressing. The PRS waves gradually returned in this situation. Therefore, the PRS phenomenon seems to depend, in some unknown way, on the operant response which delivers the reward. A tentative explanation was suggested in terms of work augmenting the reward value of a reinforcer.

Cortical evoked potentials and synchronization of electrocortical activity during consummatory behavior in rats

Electrocortical activity and averaged evoked potentials to photic stimuli were studied in rats during bar-pressing for liquid reinforcement on a fixed-ratio schedule. Reinforcing licking activity was associated with enhancement of late components of visual evoked responses and increased incidence of ‘afterdischarges’ relative to evoked responses obtained during bar-pressing. Licking activity also often resulted in apparent electrocortical synchrony at a frequency of about 7 c/sec that followed closely the frequency pattern of licking and associated phasic EMG activity of the temporal muscles. Although no one-to-one relationship between licking and apparent electrocortical synchrony could be established, the synchronous pattern appeared to represent a muscle artifact since it was present during induced spreading depression of the cortex. Thus, while data on visual evoked responses during reinforcing licking behavior concur with observations in cats, no evidence was obtained of ‘postreinforcement synchronization’ in rats.

Diffuse light input and quality of reward determine the occurrence of ‘Reward Contingent Positive Variation’ (RCPV) in cat

The effect of unpatterned light input on ECoG patterns and epicortical steady potentials were investigated in cats trained to press a lever for 0.8–1.0 ml of milk reward. High voltage (120–200 μV) bursts of post-reinforcement synchronization (PRS) with an average frequency of 7.8 c/sec were associated with epicortical positive steady potential shift, termed ‘Reward Contingent Positive Variation’ (RCPV). These ECoG phenomena occured over the striate, parastriate, and association cortex, and only in the presence of light in the test chamber, even though the subjects were trained in the absence of light. Visual perception of reward was not essential for the occurrence of PRS-RCPV phenomena, since they were observed in the same subjects whose patterned vision was totally prevented by translucent contact lenses or goggles. The role of diffuse light input (devoid of any conceivable conditional properties) in synaptic mechanisms of ECoG synchronization has been discussed. It is suggested that during instrumentally conditioned behavior, in constrast to a relaxed state and sleep onset, diffuse light input is necessary for triggering the intracortical phasing mechanism of neuronal activity which is presumably based on widespread and synchronized recurrent hyperpolarizing inhibition of large population on neurons. If this interpretation is correct, the epicortical RCPV can be visualized as resulting from electrotonic spread of the hyperpolarization of neuronal soma to apical dendrites. Since the PRS-RCPV responses are not restricted to striate and parastriate cortex in alert subjects 7,25–27, it is postulated that diffuse light input supplies the ‘electromotive energy’ for inhibitory mechanisms in cortical regions involved in processing of both visual and non-visual input.

Steady potential correlates of positive reinforcement and sleep onset in the cat; ‘reward contingent positive variation’ (RCPV)

Summary Nine cats were implanted with epidural non-polarizable Ag-AgCl electrodes over the parieto-occipital and frontal cortex. In addition, similar electrodes were implanted in subjacent white matter of the posterior marginal, medial suprasylvian, and anterior ectosylvian gyri for transcortical recording. Two bone marrow reference electrodes were also used: one in the posterior aspect of the bone overlaying the frontal sinus, and another in the occipital crest. In ‘acute’ anesthetized preparations it was found that the transcortical recording was free from contamination caused by eye-motion potentials. The chronically implanted subjects, kept on 23 h food and water deprivation schedule, were trained to press a lever for 0.5–1 ml of milk reward. Potentials caused by lapping or licking were recorded from an electrode implanted at the bottom of the frontal sinus or in the temporal bone. Eye movements were monitored from an electrode placed in the orbital bone immediately behind the eyeball. After 2–3 weeks of training, all subjects displayed bursts of high-voltage slow-wave activity of 5–9.5 c/sec over the primary and secondary visual projections during consumption of milk reward i.e. , characteristic post-reinforcement synchronization (PRS). The PRS activity was always associated with 200–700 μV positive steady potential (SP) shift over the visual cortex in either transcortical or surface bipolar recording with reference to the anterior ectosylvian gyrus or frontal sinus. This SP shift was contingent upon the presence of light in the test chamber, although the subjects were trained and habituated to perform in the dark; the SP shift was also contingent upon the relative quality and taste of food reward. Therefore, it was termed ‘reward contingent positive variation’ (RCPV). Eye movements, which in several instances could be correlated with relatively small (30–60 μV) positive lambda waves over the visual cortex, did not significantly contribute to the RCPV. In satiated subjects during sleep onset or grooming activity a similar phasic positive SP shift occurred over the same cortical area; it was always associated with or shortly followed by a burst of high-voltage slow-wave activity of 6–8 c/sec; the latter SP shift was termed ‘sleep onset positive variation’ (SOPV). The occurrence of the SOPV was facilitated by the presence of ambient light, but, in distinction to the RCPV responses, it was not absolutely contingent on it. Two relatively small amplitude but rather consistent mirror image reversals of RCPV and SOPV responses were observed: one in the subjacent white matter 3 mm beneath the striate and parastriate cortex, and another at the surface in the immediate periphery of the visual projections, i.e. , over the anterior portions of the ectosylvian, suprasylvian and marginal gyri. The phasic occurrence of SOPV immediately prior to the development of delta slow-wave sleep patterns resulted usually in a cumulative and more stable positive SP shift over the striate cortex. During the peak of a cumulative SOPV a potential difference of approximately 600–700 μV was observed, when measured epidurally across the parieto-occipital cortex between the anterior ectosylvian and posterior marginal gyri. This potential became more diffuse and partially dissipated with the development of slow-wave delta sleep patterns. The PRS, RCPV and SOPV phenomena have been interpreted as processes subserving internal inhibition.

Positive reinforcement and visual evoked potentials in cat

Visual evoked potentials (EPs) to light flashes or to central electrical stimuli were studied in freely moving rats trained to press a lever for milk reward. The EPs were recorded from the optic chiasm (OC), dorsal lateral geniculate nucleus (LG) and striate cortex (posterior marginal gyrus) during the following behavioral states: (1) non-rewarded lever pressing performance (NOR); (2) consumption of reward; (3) relaxed wakefulness after satiation (REL); (4) sleep onset; and (5) emotional excitement (EXC). Light flashes were delivered from a tiny contact flash unit placed on the cornea. Both pupils were maximally dilated with homatropine. The EPs were studied either in absence or in the presence of ambient light entering the nonstimulated eye. A minimum intensity of ambient light entering one eye during consumption of reward was necessary for a bilateral occurence of high amplitude 6–9 c/sec rhythmic ECoG activity over the striate cortex, i.e., post-reinforcement synchronization (PRS). During NOR and EXC the EPs recorded from the OC, LG, and cortex (CX) were poorly developed, whereas those during PRS, REL or sleep onset were conspicuously augmented. The increase in amplitude of the OC and LG EPs was associated with increase in latency of the primary positive peaks. The presence of ambient light entering the non-stimulated eye during consumption of reward, associated with PRS, had a facilitatory effect on the EPs in the OC and LG. During other behavioral states: NOR, EXC, REL or sleep onset, ambient light had no significant effect. These observations indicate that a central interaction of light and gustatory input results in centrifugal modulation of gross retinal output. In contrast to flash-induced cortical EPs, those elicited centrally by electrical stimulation of the LG were markedly suppressed during PRS, REL or sleep onset as compared to EPs recorded during NOR or EXC. The facilitation of cortical EPs to LG stimulation observed during NOR and EXC was markedly potentiated by ipsilateral and contralateral input of ambient light. On the other hand, bilateral input of ambient light increased the amplitude and duration of PRS activity and suppressed the EPs, thus sharpening the contrast between the EPs observed during NOR or EXC and those recorded during PRS.

Postreinforcement synchronization: Quantitative analysis of amphetamine effects

Low doses of amphetamine (.25,.50, & 1.00mg/kg up.) produced decrements in postreinforcement synchronization (PRS), but increased high voltage activity during nondrinking conditions. At 2.00 mg/kg the drug caused refusal to drink together with an increase in low-voltage activity during a nondrinking period. The complex array of amphetamine effects on ECG are discussed.

Post-reinforcement electrocortical synchronization and facilitation of cortical somato-sensory evoked potentials during instrumentally conditioned appetitive behavior in the cat

Five cats bearing chronically implanted cortical electrodes were trained to press a lever for 0.5 ml of milk reward. They displayed characteristic bursts of high voltage slow wave (6–9 c/sec) activity; i.e., post-reinforcement synchronization (PRS) over the visual cortex (posterior marginal gyrus) after the presentation and during consumption of milk reward. Computer averaged somato-sensory evoked potentials (EPs) induced by mild electrical stimuli delivered to the skin were recorded over the specific and non-specific cortical projections during the following behavioral states: (1) after non-rewarded lever pressing (NOR); (2) after presentation of reward in the illuminated environment; i.e., during bursts of PRS; (3) during consumption of reward in the dark (RD); i.e., during complete suppression of PRS activity; (4) in a waking but relaxed subject after satiation (REL); (5) during emotional excitement (EXC); and (6) during the early stage of spindle slow wave sleep (SL). The subjects were habituated to mild skin stimuli whose intensity was kept 25% below the threshold which interfered with the occurrence of PRS. These stimuli were neither disrupting the subject's performance nor physiological sleep. The irrelevant character of the stimuli was maintained by their presenting at random during aforementioned six behavioral states. Most of the fourteen EPs (comprising 9 primary and 14 secondary components) recorded over frontal and parieto-occipital cortex were conspicuously augmented during bursts of PRS and during SL as compared to the corresponding EPs obtained during NOR, EXC, REL and RD. The primary responses in all PRS EPs and in six SL EPs were larger than those obtained during NOR or EXC; they were also larger in six PRS EPs as compared to those recorded during SL (two of them showed no difference and one PRS EPs was smaller); five PRS EPs were larger than those obtained during REL. All fourteen secondary responses showed a conspicuous augmentation during SL, PRS and REL as compared to the corresponding responses obtained during NOR and EXC. The amplitude of eight PRS EPs was comparable to that obtained during SL (the remaining 6 were smaller). All except one PRS EPs were larger than REL EPs (one showed no difference). Twelve PRS EPs were significantly larger than the corresponding ones recorded during RD suggesting that the visual input may play a role in the reward-induced facilitation of long latency somatosensory cortical EPs recorded over primary and secondary visual projections, and even beyond the classical visual cortex over the anterior marginal, and anterior and posterior sigmoid gyri. A possible physiological significance of the phasic facilitation of “irrelevant” sensory input caused by positive reinforcement has been discussed.

Post-reinforcement electrocortical synchronization and facilitation of cortical auditory evoked potentials in appetitive instrumental conditioning

Food deprived cats trained to press a lever on a variable interval schedule for milk reward displayed bursts of relatively large amplitude spindle ECoG activity topographically restricted to the parieto-occipital region during the consumption of reward. An attempt was made to explore the mechanisms involved in these ECoG phenomena known as post-reinforcement synchronization (PRS). When each lever press was rewarded, the parieto-occipital cortex displayed almost continuous spindle bursts whereas the somato-sensory cortex remained desyncronized. In trained subjects both milk or water produced identical PRS. However, when water was substituted for milk during the course of an experiment, PRS was completely suppressed and overt signs of emotionality appeared. Subjects allowed to drink milk from a bowl displayed a similar topographically restricted ECoG synchronization. Both types of consummatory ECoG synchronization were suppressed in darkness or by novel environmental stimuli, and both were indiscernible from sleep-spindle ECoG activity recorded from the same region in satiated subjects, although during sleep the occurrence of spindles did not depend on the presence of light and their cortical distribution was more diffuse. Subjects were habituated to a tone (850 c/sec) or click which did not disrupt their performance and the occurrence of PRS or sleep. The primary and secondary evoked potentials recorded from various cortical projections during bursts of PRS were conspicuously augmented. Their amplitude, much greater than that recorded during relaxed wakefulness in satiated subjects, was comparable to that observed during spindle-slow wave sleep. The evoked potentials recorded after non-rewarded lever pressing; i.e., during desynchronized ECoG activity were poorly developed; they were comparable to those observed during emotional excitement. Considering the similarity of PRS patterns and sleep ECoG synchronization and assuming that the amplitude of auditory evoked potentials produced by an irrelevant stimulus reflects the fluctuations in the tonus of the reticular activating system (RAS), our results suggest that: (1) PRS is triggered by a transient depression of RAS; and (2) there is a functional overlap between reward centers and hypnogenic ( i.e., synchronizing) influences.

Learning motivated by electrical stimulation of the brain.

Learning motivated by electrical stimulation of the brain.