Hierarchical neural networks for the storage of correlated memories

Abstract

A class of hierarchical neural network models introduced by Dotsenko for the storage and associative recall of strongly correlated memories is studied analytically and numerically. In these models, patterns stored in higher levels of the hierarchy represent generalized categories and those stored in lower levels describe finer details. We first show that the models originally proposed by Dotsenko have a serious flaw: they are not able to detect or correct errors in categorization which may be present in the input. We then describe three different models which attempt to overcome this shortcoming of the original models. In the first model, the interaction between different levels of the hierarchy has the form of an external field conjugate to memories stored in the lower level. In the second model, a three-spin interaction term is included in addition to the usual binary interactions of the Hopfield type. The third model makes use of a time delay mechanism to induce, if necessary, transitions between memory states and their complements. Detailed analytical and numerical studies of the performance of these models are presented. Our analysis shows that all three models are able to detect and also to correct in varying degrees any error in categorization that may be present in the input pattern.