Serial Reversal Learning Research Articles

Changes in behavioral flexibility to cope with environment instability: theoretical and empirical insights from serial reversal learning experiments

Changes in behavioral flexibility to cope with environment instability: theoretical and empirical insights from serial reversal learning experiments

Serial reversal learning in nectar-feeding bats

We explored the behavioral flexibility of Commissaris’s long-tongued bats through a spatial serial reversal foraging task. Bats kept in captivity for short periods were trained to obtain nectar rewards from two artificial flowers. At any given time, only one of the flowers provided rewards and these reward contingencies reversed in successive blocks of 50 flower visits. All bats detected and responded to reversals by making most of their visits to the currently active flower. As the bats experienced repeated reversals, their preference re-adjusted faster. Although the flower state reversals were theoretically predictable, we did not detect anticipatory behavior, that is, frequency of visits to the alternative flower did not increase within each block as the programmed reversal approached. The net balance of these changes was a progressive improvement in performance in terms of the total proportion of visits allocated to the active flower. The results are compatible with, but do not depend on, the bats displaying an ability to ‘learn to learn’ and show that the dynamics of allocation of effort between food sources can change flexibly according to circumstances.

Serial reversal learning in an olfactory discrimination task in 3xTg-AD mice.

Male and female 3xTg-AD mice between 5 and 24 mo of age and their B6129F2/J wild-type controls were tested on a series of 18 olfactory discrimination and reversal tasks in an operant olfactometer. All mice learned the odor discriminations and reversals to a criterion of 85% correct, but the 3xTg-AD mice made fewer errors than the B6129F2/J mice in the odor discriminations and in the first six reversal learning tasks. Many mice showed evidence of near errorless learning, and on the reversal tasks the 3xTg-AD mice showed more instances of near errorless learning than the B6129F2/J mice. There was no evidence of an age effect on odor discrimination, but there was a decrease in errorless reversal learning in aged B6129F2/J mice. In long-term memory tests, there was an increase in the number of errors made but no genotype difference. The high level of performance indicates that the mice were able to develop a "learning to learn" strategy. The finding that the 3xTg-AD mice outperformed their littermate controls provides an example of paradoxical functional facilitation in these mice.

When the penny drops: sharks outsmart cichlids in serial reversal learning

Abstract Two species of fish were tested in a serial reversal learning task using two dimensional objects varying in colour and/or form. After reaching the learning criterion (LC) in the original discrimination task, individuals reversed for up to ten times, i.e., stimulus association was switched and animals rewarded for choosing the stimulus that had not been rewarded in the previous learning phase. This was conducted using the two stimuli green diamond (S+) vs. red square (S−) for the training of eight cichlids, and black star (S+) vs. black circle (S−) for the training of seven bamboo sharks. All fifteen animals completed the initial training phase and reversed successfully at least three times, indicating an ability to forsake a previously learned stimulus association in favour of a new one. Sharks, but not cichlids, also showed a decrease in the number of sessions needed to reach LC with an increase in the number of reversals conducted, showing that some sort of learning strategy, i.e., rule learning, had been applied. Animals that improve their performance within such learning paradigms are considered to possess more behavioural flexibility and advanced cognitive abilities than those who do not. More flexibility could specifically aid species living in complex habitats, social groups and subjected to changing environmental conditions.

Using repeatability of performance within and across contexts to validate measures of behavioral flexibility.

Research into animal cognitive abilities is increasing quickly and often uses methods where behavioral performance on a task is assumed to represent variation in the underlying cognitive trait. However, because these methods rely on behavioral responses as a proxy for cognitive ability, it is important to validate that the task structure does, in fact, target the cognitive trait of interest rather than non-target cognitive, personality, or motivational traits (construct validity). Although it can be difficult, or impossible, to definitively assign performance to one cognitive trait, one way to validate that task structure is more likely to elicit performance based on the target cognitive trait is to assess the temporal and contextual repeatability of performance. In other words, individual performance is likely to represent an inherent trait when it is consistent across time and across similar or different tasks that theoretically test the same trait. Here, we assessed the temporal and contextual repeatability of performance on tasks intended to test the cognitive trait behavioral flexibility in great-tailed grackles (Quiscalus mexicanus). For temporal repeatability, we quantified the number of trials to form a color preference after each of multiple color reversals on a serial reversal learning task. For contextual repeatability, we then compared performance on the serial color reversal task to the latency to switch among solutions on each of two different multi-access boxes. We found that the number of trials to form a preference in reversal learning was repeatable across serial color reversals and the latency to switch a preference was repeatable across color reversal learning and the multi-access box contexts. This supports the idea that the reversal learning task structure elicits performance reflective of an inherent trait, and that reversal learning and solution switching on multi-access boxes similarly reflect the inherent trait of behavioral flexibility.

Environmental enrichment improves cognitive flexibility in rainbow trout in a visual discrimination task: first insights.

Research on fish cognition provides strong evidence that fish are endowed with high level cognitive skills. However, most studies on cognitive flexibility and generalization abilities, two key adaptive traits for captive animals, focused on model species, and farmed fish received too little attention. Environmental enrichment was shown to improve learning abilities in various fish species, but its influence on cognitive flexibility and generalization abilities is still unknown. We studied farmed rainbow trout (Oncorhynchus mykiss) as an aquaculture model to study how environmental enrichment impacts their cognitive abilities. Using an operant conditioning device, allowing the expression of a motivated choice, we measured fish cognitive flexibility with serial reversal learning tests, after a successful acquisition phase based on two colors discrimination (2-alternative forced choice, 2-AFC), and their ability to generalize a rewarded color to any shape. Eight fish were divided into two groups: Condition E (fish reared from fry stages under enriched conditions with plants, rocks and pipes for ~9 months); Condition B (standard barren conditions). Only one fish (condition E) failed in the habituation phase of the device and one fish (condition B) failed in the 2-AFC task. We showed that after a successful acquisition phase in which the fish correctly discriminated two colors, they all succeeded in four reversal learnings, supporting evidence for cognitive flexibility in rainbow trout. They were all successful in the generalization task. Interestingly, fish reared in an enriched environment performed better in the acquisition phase and in the reversal learning (as evidenced by fewer trials needed to reach the learning criterion), but not in the generalization task. We assume that color-based generalization may be a simpler cognitive process than discriminative learning and cognitive flexibility, and does not seem to be influenced by environmental conditions. Given the small number of individuals tested, our results may be considered as first insights into cognitive flexibility in farmed fish using an operant conditioning device, but they pave the way for future studies. We conclude that farming conditions should take into account the cognitive abilities of fish, in particular their cognitive flexibility, by allowing them to live in an enriched environment.

FreiBox: A Versatile Open-Source Behavioral Setup for Investigating the Neuronal Correlates of Behavioral Flexibility via 1-Photon Imaging in Freely Moving Mice.

To survive in a complex and changing environment, animals must adapt their behavior. This ability is called behavioral flexibility and is classically evaluated by a reversal learning paradigm. During such a paradigm, the animals adapt their behavior according to a change of the reward contingencies. To study these complex cognitive functions (from outcome evaluation to motor adaptation), we developed a versatile, low-cost, open-source platform, allowing us to investigate the neuronal correlates of behavioral flexibility with 1-photon calcium imaging. This platform consists of FreiBox, a novel low-cost Arduino behavioral setup, as well as further open-source tools, which we developed and integrated into our framework. FreiBox is controlled by a custom Python interface and integrates a new licking sensor (strain gauge lickometer) for controlling spatial licking behavioral tasks. In addition to allowing both discriminative and serial reversal learning, the Arduino can track mouse licking behavior in real time to control task events in a submillisecond timescale. To complete our setup, we also developed and validated an affordable commutator, which is crucial for recording calcium imaging with the Miniscope V4 in freely moving mice. Further, we demonstrated that FreiBox can be associated with 1-photon imaging and other open-source initiatives (e.g., Open Ephys) to form a versatile platform for exploring the neuronal substrates of licking-based behavioral flexibility in mice. The combination of the FreiBox behavioral setup and our low-cost commutator represents a highly competitive and complementary addition to the recently emerging battery of open-source initiatives.

Environmental, individual and social traits of free-ranging raccoons influence performance in cognitive testing.

Cognitive abilities, such as learning and flexibility, are hypothesized to aid behavioral adaptation to urbanization. Although growing evidence suggests that cognition may indeed facilitate persistence in urban environments, we currently lack knowledge of the cognitive abilities of many urban taxa. Recent methodological advances, including radio frequency identification (RFID), have extended automated cognitive testing into the field but have yet to be applied to a diversity of taxa. Here, we used an RFID-enabled operant conditioning device to assess the habituation, learning and cognitive flexibility of a wild population of raccoons (Procyon lotor). We examined how several biological and behavioral traits influenced participation and performance in testing. We then compared the cognitive performance of wild raccoons tested in natural conditions with that of wild-caught raccoons tested in captivity from a previous study. In natural conditions, juvenile raccoons were more likely to habituate to the testing device, but performed worse in serial reversal learning, compared with adults. We also found that docile raccoons were more likely to learn how to operate the device in natural conditions, which suggests a relationship between emotional reactivity and cognitive ability in raccoons. Although raccoons in both captive and natural conditions demonstrated rapid associative learning and flexibility, raccoons in captive conditions generally performed better, likely owing to the heightened vigilance and social interference experienced by raccoons in natural conditions. Our results have important implications for future research on urban carnivores and cognition in field settings, as well as our understanding of behavioral adaptation to urbanization and coexistence with urban wildlife.

Impairments in goal-directed action and reversal learning in a proportion of individuals with psychosis

Cognitive impairment in psychosis is one of the strongest predictors of functional decline. Problems with decision-making processes, such as goal-directed action and reversal learning, can reflect cortico-striatal dysfunction. The heterogenous symptoms and neurobiology observed in those with psychosis suggests that specific cognitive phenotypes may reflect differing causative mechanisms. As such, decision-making performance could identify subgroups of individuals with more severe cortico-striatal dysfunction and help to predict their functional decline. The present work evaluated the relationship between goal-directed action, reversal learning, and symptom profiles in those with psychosis. We assessed decision-making processes in healthy controls (N = 34) and those with persistent psychosis (N = 45), subclassifying subjects based on intact/impaired goal-directed action. Compared with healthy controls (<20%), a large proportion (58%) of those with persistent psychosis displayed impaired goal-directed action, predicting poor serial reversal learning performance. Computational approaches indicated that those with impaired goal-directed action had a decreased capacity to rapidly update their prior beliefs in the face of changing contingencies. Impaired decision-making also was associated with reduced levels of grandiosity and increased problems with abstract thinking. These findings suggest that prominent decision-making deficits, indicative of cortico-striatal dysfunction, are present in a large proportion of people with persistent psychosis. Moreover, these impairments would have significant functional implications in terms of planning and abstract thinking.

Domestic hens succeed at serial reversal learning and perceptual concept generalisation using a new automated touchscreen device

Improving the welfare of farm animals depends on our knowledge on how they perceive and interpret their environment; the latter depends on their cognitive abilities. Hence, limited knowledge of the range of cognitive abilities of farm animals is a major concern. An effective approach to explore the cognitive range of a species is to apply automated testing devices, which are still underdeveloped in farm animals. In screen-like studies, the uses of automated devices are few in domestic hens. We developed an original fully automated touchscreen device using digital computer-drawn colour pictures and independent sensible cells adapted for cognitive testing in domestic hens, enabling a wide range of test types from low to high complexity. This study aimed to test the efficiency of our device using two cognitive tests. We focused on tasks related to adaptive capacities to environmental variability, such as flexibility and generalisation capacities as this is a good start to approach more complex cognitive capacities. We implemented a serial reversal learning task, categorised as a simple cognitive test, and a delayed matching-to-sample (dMTS) task on an identity concept, followed by a generalisation test, categorised as more complex. In the serial reversal learning task, the hens performed equally for the two changing reward contingencies in only three reversal stages. In the dMTS task, the hens increased their performance rapidly throughout the training sessions. Moreover, to the best of our knowledge, we present the first positive result of identity concept generalisation in a dMTS task in domestic hens. Our results provide additional information on the behavioural flexibility and concept understanding of domestic hens. They also support the idea that fully automated devices would improve knowledge of farm animals’ cognition.

Serial visual reversal learning in harbor seals (Phoca vitulina)

Progressively improving performance in a serial reversal learning (SRL) test has been associated with higher cognitive abilities and has served as a measure for cognitive/behavioral flexibility. Although the cognitive and sensory abilities of marine mammals have been subject of extensive investigation, and numerous vertebrate and invertebrate species were tested, SRL studies in aquatic mammals are sparse. Particularly in pinnipeds, a high degree of behavioral flexibility seems probable as they face a highly variable environment in air and underwater. Thus, we tested four harbor seals in a visual two-alternative forced-choice discrimination task and its subsequent reversals. We found significant individual differences in performance. One individual was able to solve 37 reversals showing progressive improvement of performance with a minimum of 6 errors in reversal 33. Two seals mastered two reversals, while one animal had difficulties in learning the discrimination task and failed to complete a single reversal. In conclusion, harbor seals can master an SRL experiment; however, the performance is inferior to results obtained in other vertebrates in comparable tasks. Future experiments will need to assess whether factors such as the modality addressed in the experiment have an influence on reversal learning performance or whether indeed, during evolution, behavioral flexibility has not specifically been favored in harbor seals.

Dissociable roles for the striatal cholinergic system in different flexibility contexts.

The production of behavioural flexibility requires the coordination and integration of information from across the brain, by the dorsal striatum. In particular, the striatal cholinergic system is thought to be important for the modulation of striatal activity. Research from animal literature has shown that chemical inactivation of the dorsal striatum leads to impairments in reversal learning. Furthermore, proton magnetic resonance spectroscopy work has shown that the striatal cholinergic system is also important for reversal learning in humans. Here, we aim to assess whether the state of the dorsal striatal cholinergic system at rest is related to serial reversal learning in humans. We provide preliminary results showing that variability in choline in the dorsal striatum is significantly related to both the number of perseverative and regressive errors that participants make, and their rate of learning from positive and negative prediction errors. These findings, in line with previous work, suggest the resting state of dorsal striatal cholinergic system has important implications for producing flexible behaviour. However, these results also suggest the system may have heterogeneous functionality across different types of tasks measuring behavioural flexibility. These findings provide a starting point for further interrogation into understanding the functional role of the striatal cholinergic system in flexibility.

Assessing human performance during contingency changes and extinction tests in reversal-learning tasks.

Serial reversal-learning procedures are simple preparations that allow for a better understanding of how animals learn about environmental changes, including flexibly shifting responding to adapt to changing reinforcement contingencies. The present study examined serial reversal learning with humans by arranging both midsession and variable contingency reversals across two experiments. We also examined the effects of extinction by adding nonreinforced trials at the end of later sessions and provided the first evaluation of effects of win-stay/lose-shift versus counting strategies on accuracy and response latency of humans' reversal-learning performance. In each experiment, responding tracked contingency reversals, primarily with participants using either win-stay/lose-shift or counting strategies. Introducing variable reversal points in the second experiment resulted in near-exclusive win-stay/lose-shift responding among participants and eliminated counting of trials. Each experiment also revealed an immediate shift from S2 to S1 after experiencing extinction during the initial test trial, indicating resurgence of the initial response through a win-stay/lose-shift response pattern. Therefore, the present study replicates and extends prior findings of a win-stay/lose shift response pattern in situations of greater uncertainty. These findings suggest that differences in environmental certainty induce qualitatively different decision-making strategies.

A rapid serial reversal learning assessment for age-related cognitive deficits in pet dogs

Assessments for behavioral inhibition in pet dogs that can rapidly detect age-related cognitive deficits (ARCD) using inexpensive and accessible materials may aid in diagnosing canine dementia and may facilitate translational research on Alzheimer’s disease in humans. In this study, we designed and deployed a spatial serial reversal learning test in which 80 pet dogs were required to learn which of two identical boxes contained a hidden food treat. Each time the dog chose the correct box in three consecutive trials the procedure was repeated using the other box. All dogs that completed shaping (n = 62) also completed the 30-minute assessment. Middle-aged dogs chose the correct box more often than younger and older dogs. This cognitive decline was detectable with a stand-alone score for perseveration that can be easily measured and interpreted by clinicians and dog owners. Age did not predict how frequently the dog learned the serially-reversing reward contingency but older and younger dogs displayed longer streaks of perseverative errors. Thus, ARCD in dogs may be better characterized by bouts of severe cognitive dysfunction rather than temporally-consistent cognitive deficits. We suggest that future ARCD assessments for pet dogs should include measurements for intra-individual variability.

Glutamatergic and Serotonergic Modulation of Rat Medial and Lateral Orbitofrontal Cortex in Visual Serial Reversal Learning.

Adapting behavior to a dynamic environment requires both steadiness when the environment is stable and behavioral flexibility in response to changes. Much evidence suggests that cognitive flexibility, which can be operationalized in reversal learning tasks, is mediated by cortico-striatal circuitries, with the orbitofrontal cortex (OFC) playing a prominent role. The OFC is a functionally heterogeneous region, and we have previously reported differential roles of lateral (lOFC) and medial (mOFC) regions in a touchscreen serial visual reversal learning task for rats using pharmacological inactivation. Here, we investigated the effects of pharmacological overactivation of these regions using a glutamate transporter 1 (GLT-1) inhibitor, dihydrokainate (DHK), which increases extracellular glutamate by blocking its reuptake. We also tested the impact of antagonism of the serotonin 2A receptor (5-HT2AR), which modulates glutamate action, in the mOFC and lOFC on the same task. Overactivation induced by DHK produced dissociable effects in the mOFC and lOFC, with more prominent effects in the mOFC, specifically improving performance in the early, perseveration phase. Intra-lOFC DHK increased the number of omitted responses without affecting errors. In contrast, blocking the 5-HT2AR in the lOFC impaired reversal learning overall, while mOFC 5-HT2AR blockade had no effect. These results further support dissociable roles of the rodent mOFC and lOFC in deterministic visual reversal learning and indicate that modulating glutamate transmission through blocking the GLT-1 and the 5-HT2AR have different roles in these two structures.

Brain size does not predict learning strategies in a serial reversal learning test.

ABSTRACTReversal learning assays are commonly used across a wide range of taxa to investigate associative learning and behavioural flexibility. In serial reversal learning, the reward contingency in a binary discrimination is reversed multiple times. Performance during serial reversal learning varies greatly at the interspecific level, as some animals adopt a rule-based strategy that enables them to switch quickly between reward contingencies. A larger relative brain size, generating enhanced learning ability and increased behavioural flexibility, has been proposed to be an important factor underlying this variation. Here, we experimentally tested this hypothesis at the intraspecific level. We used guppies (Poecilia reticulata) artificially selected for small and large relative brain size, with matching differences in neuron number, in a serial reversal learning assay. We tested 96 individuals over 10 serial reversals and found that learning performance and memory were predicted by brain size, whereas differences in efficient learning strategies were not. We conclude that variation in brain size and neuron number is important for variation in learning performance and memory, but these differences are not great enough to cause the larger differences in efficient learning strategies observed at higher taxonomic levels.

Supplemental Material for Glutamatergic and Serotonergic Modulation of Rat Medial and Lateral Orbitofrontal Cortex in Visual Serial Reversal Learning

Supplemental Material for Glutamatergic and Serotonergic Modulation of Rat Medial and Lateral Orbitofrontal Cortex in Visual Serial Reversal Learning

Serial reversal learning in freshwater stingrays (Potamotrygon motoro).

Serial reversal learning is considered a reliable approach for the testing of behavioral flexibility, and animals that inhabit fluctuating habitats and different environments are expected to possess behavioral and cognitive flexibility. The ocellate river stingray (Potamotrygon motoro) is one such species. Comprising the first serial reversal learning experiment among elasmobranchs, this study trained seven juvenile P. motoro in a visual two-alternative forced-choice task, in which a food-rewarded stimulus and an unrewarded alternative stimulus were presented in pseudo-random order on either side of a barrier. In the session after a stingray reached the learning criterion (LC), food was associated with the alternative stimulus, and this reward association continued to be switched whenever LC was achieved. Overall, five stingrays reversed successfully at least once. All of them required more sessions (mean = 67.8 sessions) for the first reversal than they needed during training (mean = 22 sessions). One stingray demonstrated progressive improvement across four reversal phases, thus showing that Potamotrygon motoro can inhibit a previously learned association faster with experience, probably as it develops strategies for rule identification. One individual became slower to reach the (LC) across three reversal phases, which may indicate a dominant influence of proactive interference. Another stingray demonstrated large fluctuations across three reversals. The fourth individual completed two reversals and required approximately the same number of sessions for each. One stingray had just completed one reversal by the end of this study. Since stingrays took longer than a related species to reverse learning, future studies could look at the effects of changing stimulus type or training regime on the performance of P. motoro.

Visual Discrimination, Serial Reversal, and Extinction Learning in the mdx Mouse.

Duchenne muscular dystrophy (DMD) is the most common form of muscular dystrophy and the most common neuromuscular disorder. In addition to neuromuscular consequences, some individuals with DMD experience global intellectual dysfunction and executive dysfunction of unknown mechanistic origin. The cognitive profile of the mdx mouse, the most commonly used mouse model of DMD, has been incompletely characterized and has never been assessed using the touchscreen operant conditioning paradigm. The touchscreen paradigm allows the use of protocols that are virtually identical to those used in human cognitive testing and may, therefore, provide the most translational paradigm for quantifying mouse cognitive function. In the present study, we used the touchscreen paradigm to assess the effects of the mdx mutation on visual discrimination learning, serial reversal learning, and extinction learning. To enable measuring task-dependent learning and memory processes while holding demands on sensory-driven information processing constant, we developed equally salient visual stimuli and used them on all experimental stages. Acquisition of the initial pairwise visual discrimination was facilitated in mdx mice relative to wildtype littermates; this effect was not explained by genotypic differences in impulsivity, motivation, or motor deficits. The mdx mutation had no effect on serial reversal or extinction learning. Together, findings from this study and previous studies suggest that mdx effects on cognitive function are task-specific and may be influenced by discrimination type (spatial, visual), reward type (food, escape from a non-preferred environment), sex, and genetic background.

Dissociable and Paradoxical Roles of Rat Medial and Lateral Orbitofrontal Cortex in Visual Serial Reversal Learning.

ABSTRACTMuch evidence suggests that reversal learning is mediated by cortico-striatal circuitries with the orbitofrontal cortex (OFC) playing a prominent role. The OFC is a functionally heterogeneous region, but potential differential roles of lateral (lOFC) and medial (mOFC) portions in visual reversal learning have yet to be determined. We investigated the effects of pharmacological inactivation of mOFC and lOFC on a deterministic serial visual reversal learning task for rats. For reference, we also targeted other areas previously implicated in reversal learning: prelimbic (PrL) and infralimbic (IL) prefrontal cortex, and basolateral amygdala (BLA). Inactivating mOFC and lOFC produced opposite effects; lOFC impairing, and mOFC improving, performance in the early, perseverative phase specifically. Additionally, mOFC inactivation enhanced negative feedback sensitivity, while lOFC inactivation diminished feedback sensitivity in general. mOFC and lOFC inactivation also affected novel visual discrimination learning differently; lOFC inactivation paradoxically improved learning, and mOFC inactivation had no effect. We also observed dissociable roles of the OFC and the IL/PrL. Whereas the OFC inactivation affected only perseveration, IL/PrL inactivation improved learning overall. BLA inactivation did not affect perseveration, but improved the late phase of reversal learning. These results support opponent roles of the rodent mOFC and lOFC in deterministic visual reversal learning.