Neural Associative Memory Research Articles

PATTERN STORAGE AND HOPFIELD NEURAL ASSOCIATIVE MEMORY WITH HIDDEN STRUCTURE

This paper is concerned with the formulation of neural associative memories. Centered around the fundamental issue of the memory storage, we examine the deficiencies associated with the standard Hopfield net. To overcome the problems, we pursue a data-driven design approach by modifying the configuration of the Hopfield net to allow hidden structures. As important results, we show how the well-known sum-of-outer product rule can be utilized to explore the freedom provided by the hidden structures leading to the desired memory performance.

Exact associative neural memory dynamics utilizing Boolean matrices

The exact dynamics of shallow loaded associative neural memories are generated and characterized. The Boolean matrix analysis approach is employed for the efficient generation of all possible state transition trajectories for parallel updated binary-state dynamic associative memories (DAMs). General expressions for the size of the basin of attraction of fundamental and oscillatory memories and the number of oscillatory and stable states are derived for discrete synchronous Hopfield DAMs loaded with one, two, or three even-dimensionality bipolar memory vectors having the same mutual Hamming distances between them. Spurious memories are shown to occur only if the number of stored patterns exceeds two in an even-dimensionality Hopfield memory. The effects of odd- versus even-dimensionality memory vectors on DAM dynamics and the effects of memory pattern encoding on DAM performance are tested. An extension of the Boolean matrix dynamics characterization technique to other, more complex DAMs is presented.

Linear algebra approach to neural associative memories and noise performance of neural classifiers

The authors present an analytic evaluation of saturation and noise performance for a large class of associative memories based on matrix operations. The importance of using standard linear algebra techniques for evaluating noise performance of associative memories is emphasized. The authors present a detailed comparative analysis of the correlation matrix memory and the generalized inverse memory construction rules for auto-associative memory and neural classifiers. Analytic results for the noise performance of neural classifiers that can store several prototypes in one class are presented. The analysis indicates that for neural classifiers the simple correlation matrix memory provides better noise performance than the more complex generalized inverse memory. >

A Neural Net Associative Memory for Real-Time Applications

A parallel hardware implementation of the associative memory neural network introduced by Hopfield is described. The design utilizes the Geometric Arithmetic Parallel Processor (GAPP), a commercially available single-chip VLSI general-purpose array processor consisting of 72 processing elements. The ability to cascade these chips allows large arrays of processors to be easily constructed and used to implement the Hopfield network. The memory requirements and processing times of such arrays are analyzed based on the number of nodes in the network and the number of exemplar patterns. Compared with other digital implementations, this design yields significant improvements in runtime performance and offers the capability of using large neural network associative memories in real-time applications.

Shaping attraction basins in neural networks

An interesting duality between two formally related schemes for neural associative memory is exploited to shape the attraction basins of stored memories. Considered are a family of spectral algorithms—based on specifying the spectrum of the matrix of weights as a function of the memories to be stored—and a class of dual spectral algorithms—based on manipulations of the orthogonal subspace of the memories, which are expanded here. These algorithms are shown to attain near maximal memory storage capacity of the order of n, and are shown to typically require the order of n 3 elementary operations for their implementation. Signal-to-noise ratio arguments are presented showing a duality in the error-correction behaviour of the two schemes: the spectral algorithm demonstrates memory-specific attraction around the memories, while the dual spectral algorithm demonstrates direction-specific attraction. Composite algorithms capable of joint memory-specific and direction-specific attraction are presented as a means of variably shaping attraction basins around desired memories. Computer simulations are included in support of the analysis.

Optimal learning in neural network memories

The authors examine general learning procedures for neural network associative memories and find algorithms which optimise convergence.

High Performance Recording Algorithm For Hopfield Model Associative Memories

A new recording technique for Hopfield-type dynamic auto-associative memories is proposed. The new technique is based on the finite and exponentially convergent algorithm of Ho and Kashyap for the solution of a system of linear inequalities. Associative neural memories recorded with the proposed algorithm are shown to be superior to those recorded with the Hopfield outer-product and Kohonen generalized-inverse techniques. The new recording algorithm is characterized by high capacity, high convergence rates to stored memories and low convergence rates to false and oscillatory states. The issue of stable false and oscillatory states is raised, and it is shown that such states appear to have a direct Boolean logic relationship with the stored memories.

Fault tolerance of neural associative memories

The effects of hardware limitations and fabrication faults on the fault tolerance of neural associative memories using the Hopfield interconnect topology are investigated. It is shown that neural computing structures are not by definition fault tolerant, and that the degree of tolerance is very sensitive to the assumed physical fault model and to the nature of the stored information.

A principle of neural associative memory

This paper presents a demonstration of a possible mechanism of distributed memory which by association recalls the missing part of a memorized pattern. A model based upon a hypothesis of synaptic modification is constructed, and its behaviour is analysed with the aid of computer simulations. The hypothesis implies a change in the transmission etficacies of synapses in proportion to both pre- and postsynaptic activity levels. The basic design of this model is applicable to different types of neural networks, but is here exemplified in an idealized cerebral cortex. Incorporation of the principle of lateral inhibition implies that the selectivity of the memory mechanism is made greatly superior to that of many other models of associative memory. This has made it practicable to demonstrate the selective recall of a large number of signal patterns from an associative memory network based upon assumptions that are anatomically and physiologically feasible. A computer simulation with photographic images (faces) is reported in which one of 500 stored images was faithfully recalled when a fragment of it was used as the key excitation.