Partially annealed neural networks

Abstract

We consider the Hopfield model of neural networks in which the patterns, as well as the spins, are dynamical variables. The characteristic time scales of the dynamics of the spins and the patterns are assumed to be widely separated such that the spins completely equilibrate at the time scale at which the elementary changes in the patterns take place. We study the situation in which each type of variable thermalizes at different temperatures, respectively, T and T'. In this case, such a system is described in terms of the traditional replica formalism in which the number of replicas n=T/T' is still the finite parameter. The complete phase diagram of the model in the space of the parameters T, alpha and n is obtained. If the parameter n is negative, the model is argued to present some similarities with the unlearning training algorithm. In this case a substantial increase in size of the retrieval phase in the plane (T, alpha ) is found.