Recovery of Unrehearsed Items in Connectionist Models

Abstract

When gradient-descent models with hidden units are retrained on a portion of a previously learned set of items, performance on both the relearned and unrelearned items improves. Previous explanations of this phenomenon have not adequately distinguished recovery, which is dependent on original learning, from generalization, which is independent of original learning. Using a measure of vector similarity to track global changes in the weight state of three-layer networks, we show that (a) unlike in networks without hidden units, recovery occurs in the absence of generalization in networks with hidden units, and (b) when the conditions of learning are varied, changes in the extent of recovery are reflected in changes in the extent to which the weights move back towards their values held after original learning. The implications of this work for rehabilitation studies, human relearning and models of human long-term memory are also considered.