Training the Gradient Field of a Dynamic Hopfield (Recurrent) Network for Classification

Abstract

This chapter discusses the problem of using a continuous recurrent neural network for associative memory. Problems with using this net for associative memory include failure to converge or slow convergence, spurious at tractors, and low storage capacity. The gradient of the state space is trained over a set of sample points including both fixed points and initial conditions. It does not require the storage and computation time of trajectory training, yet provides finer control over the basin of attraction boundaries than training for the fixed points alone. There are two main methods that have already been developed for training the gradient field of a state space of a continuous time recurrent neural network. One trains only for the fixed points and the other trains for trajectories in the state space. An algorithm for determining the basins of attraction in the network state space indirectly through training the gradient field of the network was given and a simulation performed. An analysis of what region boundaries are possible for a given net needs to be done. Simulations need to be done on situations with a larger number of categories and units, where the number of units is much larger than the number of categories.