Sparsification from dilute connectivity in a neural network model of memory

Abstract

Several hypotheses concerning implementations of associative memory in the brain rely on analyses of the capabilities of simple network models. However, the low connectivity of cerebral networks imposes constraints which sometimes do not arise clearly from such analyses. We investigate an aspect of a simple, dilute network's operation that is sometimes overlooked, namely the setting of activation thresholds. An examination of several criteria for optimal threshold assignment affords several new insights. It becomes apparent that the network's capacity (which is simply derived) is insufficient to characterize the quality of its performance. We derive the degree of ‘sparsification’ or decrease in firing probability that arises from dilution, and also the consequent losses in representational ability, and propose that they should also be taken into account. To evaluate the model's performance and suitability, we argue that one should explicitly consider the trade-off that exists between storage of patterns and preservation of information, and its consequent constraints.