Complementary Learning Systems Research Articles

Integration of new information in memory: new insights from a complementary learning systems perspective.

According to complementary learning systems theory, integrating new memories into the neocortex of the brain without interfering with what is already known depends on a gradual learning process, interleaving new items with previously learned items. However, empirical studies show that information consistent with prior knowledge can sometimes be integrated very quickly. We use artificial neural networks with properties like those we attribute to the neocortex to develop an understanding of the role of consistency with prior knowledge in putatively neocortex-like learning systems, providing new insights into when integration will be fast or slow and how integration might be made more efficient when the items to be learned are hierarchically structured. The work relies on deep linear networks that capture the qualitative aspects of the learning dynamics of the more complex nonlinear networks used in previous work. The time course of learning in these networks can be linked to the hierarchical structure in the training data, captured mathematically as a set of dimensions that correspond to the branches in the hierarchy. In this context, a new item to be learned can be characterized as having aspects that project onto previously known dimensions, and others that require adding a new branch/dimension. The projection onto the known dimensions can be learned rapidly without interleaving, but learning the new dimension requires gradual interleaved learning. When a new item only overlaps with items within one branch of a hierarchy, interleaving can focus on the previously known items within this branch, resulting in faster integration with less interleaving overall. The discussion considers how the brain might exploit these facts to make learning more efficient and highlights predictions about what aspects of new information might be hard or easy to learn. This article is part of the Theo Murphy meeting issue 'Memory reactivation: replaying events past, present and future'.

Generative Continual Concept Learning

After learning a concept, humans are also able to continually generalize their learned concepts to new domains by observing only a few labeled instances without any interference with the past learned knowledge. In contrast, learning concepts efficiently in a continual learning setting remains an open challenge for current Artificial Intelligence algorithms as persistent model retraining is necessary. Inspired by the Parallel Distributed Processing learning and the Complementary Learning Systems theories, we develop a computational model that is able to expand its previously learned concepts efficiently to new domains using a few labeled samples. We couple the new form of a concept to its past learned forms in an embedding space for effective continual learning. Doing so, a generative distribution is learned such that it is shared across the tasks in the embedding space and models the abstract concepts. This procedure enables the model to generate pseudo-data points to replay the past experience to tackle catastrophic forgetting.

A complementary learning systems approach to temporal difference learning

Complementary Learning Systems (CLS) theory suggests that the brain uses a ’neocortical’ and a ’hippocampal’ learning system to achieve complex behaviour. These two systems are complementary in that the ’neocortical’ system relies on slow learning of distributed representations while the ’hippocampal’ system relies on fast learning of pattern-separated representations. Both of these systems project to the striatum, which is a key neural structure in the brain’s implementation of Reinforcement Learning (RL). Current deep RL approaches share similarities with a ’neocortical’ system because they slowly learn distributed representations through backpropagation in Deep Neural Networks (DNNs). An ongoing criticism of such approaches is that they are data inefficient and lack flexibility. CLS theory suggests that the addition of a ’hippocampal’ system could address these criticisms. In the present study we propose a novel algorithm known as Complementary Temporal Difference Learning (CTDL), which combines a DNN with a Self-Organizing Map (SOM) to obtain the benefits of both a ’neocortical’ and a ’hippocampal’ system. Key features of CTDL include the use of Temporal Difference (TD) error to update a SOM and the combination of a SOM and DNN to calculate action values. We evaluate CTDL on Grid World, Cart–Pole and Continuous Mountain Car tasks and show several benefits over the classic Deep Q-Network (DQN) approach. These results demonstrate (1) the utility of complementary learning systems for the evaluation of actions, (2) that the TD error signal is a useful form of communication between the two systems and (3) that our approach extends to both discrete and continuous state and action spaces.

Robust spatial memory maps encoded by networks with transient connections.

The spiking activity of principal cells in mammalian hippocampus encodes an internalized neuronal representation of the ambient space—a cognitive map. Once learned, such a map enables the animal to navigate a given environment for a long period. However, the neuronal substrate that produces this map is transient: the synaptic connections in the hippocampus and in the downstream neuronal networks never cease to form and to deteriorate at a rapid rate. How can the brain maintain a robust, reliable representation of space using a network that constantly changes its architecture? We address this question using a computational framework that allows evaluating the effect produced by the decaying connections between simulated hippocampal neurons on the properties of the cognitive map. Using novel Algebraic Topology techniques, we demonstrate that emergence of stable cognitive maps produced by networks with transient architectures is a generic phenomenon. The model also points out that deterioration of the cognitive map caused by weakening or lost connections between neurons may be compensated by simulating the neuronal activity. Lastly, the model explicates the importance of the complementary learning systems for processing spatial information at different levels of spatiotemporal granularity.

The role of complementary learning systems in learning and consolidation in a quasi-regular domain

We examine the role of off-line memory consolidation processes in the learning and retention of a new quasi-regular linguistic system similar to the English past tense. Quasi-regular systems are characterized by a dominance of systematic, regular forms (e.g., walk-walked, jump-jumped) alongside a smaller number of high frequency irregulars (e.g., sit-sat, go-went), and are found across many cognitive domains, from spelling-sound mappings to inflectional morphology to semantic cognition. Participants were trained on the novel morphological system using an artificial language paradigm, and then tested after different delays. Based on a complementary systems account of memory, we predicted that irregular forms would show stronger off-line changes due to consolidation processes. Across two experiments, participants were tested either immediately after learning, 12 h later with or without sleep, or 24 h later. Testing involved generalization of the morphological patterns to previously unseen words (both experiments) as well as recall of the trained words (Experiment 2). In generalization, participants showed ‘default’ regularization across a range of novel forms, as well as irregularization for previously unseen items that were similar to unique high-frequency irregular trained forms. Both patterns of performance remained stable across the delays. Generalizations involving competing tendencies to regularize and irregularize were balanced between the two immediately after learning. Crucially, at both 12-h delays the tendency to irregularize in these cases was strengthened, with further strengthening after 24 h. Consolidated knowledge of both regular and irregular trained items contributed significantly to generalization performance, with evidence of strengthening of irregular forms and weakening of regular forms. We interpret these findings in the context of a complementary systems model, and discuss how maintenance, strengthening, and forgetting of the new memories across sleep and wake can play a role in acquiring quasi-regular systems.

Perturbation of old knowledge precedes integration of new knowledge

The importance of memory consolidation in integrating new knowledge has received much recent attention in the field of word learning. Less examined is the change in existing word knowledge as a result of learning, which we hypothesize to occur prior to the opportunity for consolidation. To test this, we had participants learn new meanings for known words and novel words. Then they performed a one-back task on a list of words that included the trained words followed by words that probed either their new or original meanings while EEGs were recorded. A probe word related to the new meaning of the preceding trained word did not show an N400 reduction compared to an unrelated word, suggesting that the new meaning had not been fully integrated, consistent with one account of complementary learning systems. However, when the probe word was related to the original meaning of the preceding trained word a perturbation effect was observed, indicated by a larger negativity at the central midline cluster (Cz) within 500–700 ms when the trained word had a new meaning than when presented as an exposure control. The perturbation effect suggests that even before a new meaning has become integrated with a word form, the attempt to learn a new meaning temporarily makes the original meaning of a word less accessible.

Dual-memory neural networks for modeling cognitive activities of humans via wearable sensors

Wearable devices, such as smart glasses and watches, allow for continuous recording of everyday life in a real world over an extended period of time or lifelong. This possibility helps better understand the cognitive behavior of humans in real life as well as build human-aware intelligent agents for practical purposes. However, modeling the human cognitive activity from wearable-sensor data stream is challenging because learning new information often results in loss of previously acquired information, causing a problem known as catastrophic forgetting. Here we propose a deep-learning neural network architecture that resolves the catastrophic forgetting problem. Based on the neurocognitive theory of the complementary learning systems of the neocortex and hippocampus, we introduce a dual memory architecture (DMA) that, on one hand, slowly acquires the structured knowledge representations and, on the other hand, rapidly learns the specifics of individual experiences. The DMA system learns continuously through incremental feature adaptation and weight transfer. We evaluate the performance on two real-life datasets, the CIFAR-10 image-stream dataset and the 46-day Lifelog dataset collected from Google Glass, showing that the proposed model outperforms other online learning methods.

Complementary learning systems within the hippocampus: a neural network modelling approach to reconciling episodic memory with statistical learning.

A growing literature suggests that the hippocampus is critical for the rapid extraction of regularities from the environment. Although this fits with the known role of the hippocampus in rapid learning, it seems at odds with the idea that the hippocampus specializes in memorizing individual episodes. In particular, the Complementary Learning Systems theory argues that there is a computational trade-off between learning the specifics of individual experiences and regularities that hold across those experiences. We asked whether it is possible for the hippocampus to handle both statistical learning and memorization of individual episodes. We exposed a neural network model that instantiates known properties of hippocampal projections and subfields to sequences of items with temporal regularities. We found that the monosynaptic pathway-the pathway connecting entorhinal cortex directly to region CA1-was able to support statistical learning, while the trisynaptic pathway-connecting entorhinal cortex to CA1 through dentate gyrus and CA3-learned individual episodes, with apparent representations of regularities resulting from associative reactivation through recurrence. Thus, in paradigms involving rapid learning, the computational trade-off between learning episodes and regularities may be handled by separate anatomical pathways within the hippocampus itself.This article is part of the themed issue 'New frontiers for statistical learning in the cognitive sciences'.

Localist models are compatible with information measures, sparseness indices, and complementary-learning systems in the brain

ABSTRACTIn this paper, I express continued support for localist modelling in psychology and critically evaluate previous studies that have sought to weaken the localist case in favour of models with thoroughgoing distributed representation. I question claims that information measures and sparseness indices derived from single-cell recording data are supportive of distributed representation and show that the patterns observed in those data can be reproduced from simulations of a model that is known to be localist. I also set out some logical objections to the complementary-learning hypothesis, particularly in as much as it is used to justify thoroughgoing distributed models of the cortex.

Does Sleep Improve Your Grammar? Preferential Consolidation of Arbitrary Components of New Linguistic Knowledge.

We examined the role of sleep-related memory consolidation processes in learning new form-meaning mappings. Specifically, we examined a Complementary Learning Systems account, which implies that sleep-related consolidation should be more beneficial for new hippocampally dependent arbitrary mappings (e.g. new vocabulary items) relative to new systematic mappings (e.g. grammatical regularities), which can be better encoded neocortically. The hypothesis was tested using a novel language with an artificial grammatical gender system. Stem-referent mappings implemented arbitrary aspects of the new language, and determiner/suffix+natural gender mappings implemented systematic aspects (e.g. tib scoiffesh + ballerina, tib mofeem + bride; ked jorool + cowboy, ked heefaff + priest). Importantly, the determiner-gender and the suffix-gender mappings varied in complexity and salience, thus providing a range of opportunities to detect beneficial effects of sleep for this type of mapping. Participants were trained on the new language using a word-picture matching task, and were tested after a 2-hour delay which included sleep or wakefulness. Participants in the sleep group outperformed participants in the wake group on tests assessing memory for the arbitrary aspects of the new mappings (individual vocabulary items), whereas we saw no evidence of a sleep benefit in any of the tests assessing memory for the systematic aspects of the new mappings: Participants in both groups extracted the salient determiner-natural gender mapping, but not the more complex suffix-natural gender mapping. The data support the predictions of the complementary systems account and highlight the importance of the arbitrariness/systematicity dimension in the consolidation process for declarative memories.

Changes in Theta and Beta Oscillations as Signatures of Novel Word Consolidation

The complementary learning systems account of word learning states that novel words, like other types of memories, undergo an offline consolidation process during which they are gradually integrated into the neocortical memory network. A fundamental change in the neural representation of a novel word should therefore occur in the hours after learning. The present EEG study tested this hypothesis by investigating whether novel words learned before a 24-hr consolidation period elicited more word-like oscillatory responses than novel words learned immediately before testing. In line with previous studies indicating that theta synchronization reflects lexical access, unfamiliar novel words elicited lower power in the theta band (4-8 Hz) than existing words. Recently learned words still showed a marginally lower theta increase than existing words, but theta responses to novel words that had been acquired 24 hr earlier were indistinguishable from responses to existing words. Consistent with evidence that beta desynchronization (16-21 Hz) is related to lexical-semantic processing, we found that both unfamiliar and recently learned novel words elicited less beta desynchronization than existing words. In contrast, no difference was found between novel words learned 24 hr earlier and existing words. These data therefore suggest that an offline consolidation period enables novel words to acquire lexically integrated, word-like neural representations.

Learning word meanings: overnight integration and study modality effects.

According to the complementary learning systems (CLS) account of word learning, novel words are rapidly acquired (learning system 1), but slowly integrated into the mental lexicon (learning system 2). This two-step learning process has been shown to apply to novel word forms. In this study, we investigated whether novel word meanings are also gradually integrated after acquisition by measuring the extent to which newly learned words were able to prime semantically related words at two different time points. In addition, we investigated whether modality at study modulates this integration process. Sixty-four adult participants studied novel words together with written or spoken definitions. These words did not prime semantically related words directly following study, but did so after a 24-hour delay. This significant increase in the magnitude of the priming effect suggests that semantic integration occurs over time. Overall, words that were studied with a written definition showed larger priming effects, suggesting greater integration for the written study modality. Although the process of integration, reflected as an increase in the priming effect over time, did not significantly differ between study modalities, words studied with a written definition showed the most prominent positive effect after a 24-hour delay. Our data suggest that semantic integration requires time, and that studying in written format benefits semantic integration more than studying in spoken format. These findings are discussed in light of the CLS theory of word learning.

Recollection and Familiarity Exhibit Dissociable Similarity Gradients: A Test of the Complementary Learning Systems Model

Memory can often be triggered by retrieval cues that are quite different from the originally encoded events, but how different memory processes respond to variations in cue-target similarity is poorly understood. We begin by presenting simulations using a neurocomputational model of recognition memory (i.e., the complementary learning systems model), which proposes that the hippocampus supports recollection of associative information whereas the surrounding cortex supports assessments of item familiarity. The simulations showed that increases in the similarity between retrieval cues and learned items led to relatively linear increases in a cortex-based memory signal but led to steeper and more thresholded increases in the hippocampal signal. We then tested the predictions of the model by examining the effects of varying cue-target similarity in two recognition memory experiments in which participants studied a list of computer-generated faces and then, at test, gave confidence and remember/know responses to morphed faces. In both experiments, as cue-target similarity was increased, familiarity-based recognition increased in a gradual and relatively linear fashion, whereas recollection showed significantly steeper gradients. The results show that recollection and familiarity exhibit distinct similarity functions in recognition memory that correspond with predicted retrieval dynamics of the hippocampus and cortex, respectively.

Stroop effects from newly learned color words: effects of memory consolidation and episodic context.

The Stroop task is an excellent tool to test whether reading a word automatically activates its associated meaning, and it has been widely used in mono- and bilingual contexts. Despite of its ubiquity, the task has not yet been employed to test the automaticity of recently established word-concept links in novel-word-learning studies, under strict experimental control of learning and testing conditions. In three experiments, we thus paired novel words with native language (German) color words via lexical association and subsequently tested these words in a manual version of the Stroop task. Two crucial findings emerged: When novel word Stroop trials appeared intermixed among native-word trials, the novel-word Stroop effect was observed immediately after the learning phase. If no native color words were present in a Stroop block, the novel-word Stroop effect only emerged 24 h later. These results suggest that the automatic availability of a novel word's meaning depends either on supportive context from the learning episode and/or on sufficient time for memory consolidation. We discuss how these results can be reconciled with the complementary learning systems account of word learning.

Neurocomputational account of memory and perception: Thresholded and graded signals in the hippocampus.

Recent evidence suggests that the hippocampus, a region critical for long-term memory, also supports certain forms of high-level visual perception. A seemingly paradoxical finding is that, unlike the thresholded hippocampal signals associated with memory, the hippocampus produces graded, strength-based signals in perception. This article tests a neurocomputational model of the hippocampus, based on the complementary learning systems framework, to determine if the same model can account for both memory and perception, and whether it produces the appropriate thresholded and strength-based signals in these two types of tasks. The simulations showed that the hippocampus, and most prominently the CA1 subfield, produced graded signals when required to discriminate between highly similar stimuli in a perception task, but generated thresholded patterns of activity in recognition memory. A threshold was observed in recognition memory because pattern completion occurred for only some trials and completely failed to occur for others; conversely, in perception, pattern completion always occurred because of the high degree of item similarity. These results offer a neurocomputational account of the distinct hippocampal signals associated with perception and memory, and are broadly consistent with proposals that CA1 functions as a comparator of expected versus perceived events. We conclude that the hippocampal computations required for high-level perceptual discrimination are congruous with current neurocomputational models that account for recognition memory, and fit neatly into a broader description of the role of the hippocampus for the processing of complex relational information.

Connectionism coming of age: legacy and future challenges.

Connectionism coming of age: legacy and future challenges.

Incorporating rapid neocortical learning of new schema-consistent information into complementary learning systems theory.

The complementary learning systems theory of the roles of hippocampus and neocortex (McClelland, McNaughton, & O'Reilly, 1995) holds that the rapid integration of arbitrary new information into neocortical structures is avoided to prevent catastrophic interference with structured knowledge representations stored in synaptic connections among neocortical neurons. Recent studies (Tse et al., 2007, 2011) showed that neocortical circuits can rapidly acquire new associations that are consistent with prior knowledge. The findings challenge the complementary learning systems theory as previously presented. However, new simulations extending those reported in McClelland et al. (1995) show that new information that is consistent with knowledge previously acquired by a putatively cortexlike artificial neural network can be learned rapidly and without interfering with existing knowledge; it is when inconsistent new knowledge is acquired quickly that catastrophic interference ensues. Several important features of the findings of Tse et al. (2007, 2011) are captured in these simulations, indicating that the neural network model used in McClelland et al. has characteristics in common with neocortical learning mechanisms. An additional simulation generalizes beyond the network model previously used, showing how the rate of change of cortical connections can depend on prior knowledge in an arguably more biologically plausible network architecture. In sum, the findings of Tse et al. are fully consistent with the idea that hippocampus and neocortex are complementary learning systems. Taken together, these findings and the simulations reported here advance our knowledge by bringing out the role of consistency of new experience with existing knowledge and demonstrating that the rate of change of connections in real and artificial neural networks can be strongly prior-knowledge dependent.

Richness of information about novel words influences how episodic and semantic memory networks interact during lexicalization

The complementary learning systems account of declarative memory suggests two distinct memory networks, a fast-mapping, episodic system involving the hippocampus, and a slower semantic memory system distributed across the neocortex in which new information is gradually integrated with existing representations. In this study, we investigated the extent to which these two networks are involved in the integration of novel words into the lexicon after extensive learning, and how the involvement of these networks changes after 24h. In particular, we explored whether having richer information at encoding influences the lexicalization trajectory. We trained participants with two sets of novel words, one where exposure was only to the words' phonological forms (the form-only condition), and one where pictures of unfamiliar objects were associated with the words' phonological forms (the picture-associated condition). A behavioral measure of lexical competition (indexing lexicalization) indicated stronger competition effects for the form-only words. Imaging (fMRI) results revealed greater involvement of phonological lexical processing areas immediately after training in the form-only condition, suggesting that tight connections were formed between novel words and existing lexical entries already at encoding. Retrieval of picture-associated novel words involved the episodic/hippocampal memory system more extensively. Although lexicalization was weaker in the picture-associated condition, overall memory strength was greater when tested after a 24hour delay, probably due to the availability of both episodic and lexical memory networks to aid retrieval. It appears that, during lexicalization of a novel word, the relative involvement of different memory networks differs according to the richness of the information about that word available at encoding.

REM Sleep Reorganizes Hippocampal Excitability

Sleep is composed of an alternating sequence of REM and non-REM episodes, but their respective roles are not known. We found that the overall firing rates of hippocampal CA1 neurons decreased across sleep concurrent with an increase in the recruitment of neuronal spiking to brief "ripple" episodes, resulting in a net increase in neural synchrony. Unexpectedly, within non-REM episodes, overall firing rates gradually increased together with a decrease in the recruitment of spiking to ripples. The rate increase within non-REM episodes was counteracted bya larger and more rapid decrease of discharge frequency within the interleaved REM episodes. Both the decrease in firing rates and the increase in synchrony during the course of sleep were correlated with the power of theta activity during REM episodes. These findings assign a prominent role of REM sleep in sleep-related neuronal plasticity.

Neurogenesis interferes with the retrieval of remote memories: Forgetting in neurocomputational terms

In contrast to models and theories that relate adult neurogenesis with the processes of learning and memory, almost no solid hypotheses have been formulated that involve a possible neurocomputational influence of adult neurogenesis on forgetting. Based on data from a previous study that implemented a simple but complete model of the main hippocampal circuitry (Weisz & Argibay, 2009), we now test this model under different situations to better study the case of remote memories. The results of this work show that following neurogenesis, the new, ongoing memories in the hippocampus are better retained than when no neurogenesis occurs at all, while the older memories are affected (to a lesser extent) by a special type of interference that is different from interference that occurs with an increasing number of memories per se. This work adds a new point of analysis in support of the interference view that might lead to the forgetting of memories in the hippocampus as they are transferred to neocortex for long-term storage, consistent with the Complementary Learning Systems models of system-level consolidation. Attention should be directed to the specific causes of interference; the results of this work signal a type of distortion of remote memories that is produced by the birth and the growth of new processing units, which results in a subtly impoverished retrieval as new neurons become active. The proposals of this model fit well with some empirical findings that are related to the issue.In the future, as new evidence emerges, we believe that this biological process, which is largely related to learning and memory, will also help to shape our ideas about normal forgetting and its possible contributions to system consolidation.