Associative Memory Networks Research Articles

Optimizing lattice-based associative memory networks by evolutionary algorithms

Abstract Problem specific network structure optimization subsumes the problem of input selection and network topology identification. Requirements to the network should be accuracy and good generalization abilities. In this contribution we describe in detail an evolutionary algorithm which performs both tasks well. Furthermore, approximation results on mathematical and real world data are presented. In this case we used lattice-based associative memory networks (LB-AMNs) using B-splines as basis functions. The method here is not restricted to B-splines as basis functions. The proposed method and algorithm can be seen as optimized classification system.

Efficient associative memory using small-world architecture

Most models of neural associative memory have used networks with broad connectivity. However, from both a neurobiological viewpoint and an implementation perspective, it is logical to minimize the length of inter-neural connections and consider networks whose connectivity is predominantly local. The “small-world networks” model described recently by Watts and Strogatz provides an interesting approach to this issue. In this paper, we show that associative memory networks with small-world architectures can provide the same retrieval performance as randomly connected networks while using a fraction of the total connection length.

Classification of cancer patients based on elemental contents of serums using bidirectional associative memory networks

Bidirectional associative memory (BAM) was applied for diagnosing cancer based on the elemental contents in serum samples. The serum samples were taken from clinical hospitals in north-east region of PR China and the elemental contents in serum samples were analyzed by inductively coupled plasma atomic emission spectrometry (ICP-AES). The elemental contents of the sample were encoded to bipolar input values for BAM computation. The BAM method was verified with independent prediction samples by using the ‘cross-validation’ method. The networks can be used to discriminate of all cancer patients from non-cancer patients at rate of 100%. A comparison study using BAM and multi-layer feed-forward neural network was made, better results were obtained using BAM networks. The effects of threshold values and output nodes of the BAM network were investigated and related problems were discussed. Results showed that the BAM would be applied to elemental analysis of serums and be promising method for diagnosis of cancer.

On discrete-time cellular neural networks for associative memories

In this paper, discrete-time cellular neural networks (DTCNNs) with a globally asymptotically stable equilibrium point, are designed to behave as associative memories. The objective is achieved by considering feedback parameters related to circulant matrices and by satisfying frequency domain stability criteria. The approach, by generating DTCNNs where the input data are fed via external inputs rather than initial conditions, enables both heteroassociative and autoassociative memories to be designed. Numerical examples are reported in order to show the capabilities of the proposed tool.

Estimation of attraction domain and exponential convergence rate of continuous feedback associative memory

We revisit the study of the attraction domain of memory patterns and exponential convergence rate of the network trajectory memory patterns for the continuous feedback associative memory by using some analysis techniques and the Lyapunov method. Some new estimation results are obtained and can be used for the evaluation of fault-tolerance capability and synthesis procedures of continuous feedback associative memory networks.

On stability of nonlinear continuous-time neural networks with delays

We utilize the Lyapunov function method to analyze stability of continuous nonlinear neural networks with delays and obtain some new sufficient conditions ensuring the globally asymptotic stability independent of delays. Three main conditions imposed on the weighting matrices are established. (i). The spectral radius ρ( M −1(| W 0|+| W τ |) K)<1. (ii). The row norm ‖ M −1(| W 0|+| W τ |) K+ P −1((| W 0|+| W τ |) KM −1) T P‖ ∞<2. (iii). μ 2( W 0)+‖ W τ ‖ 2, F <( m/ k). These three conditions are independent to each other. The delayed Hopfield network, Bidirectional associative memory network and cellular neural network are special cases of the network model considered in this paper. So we improve some previous works of other researchers.

Data based Model Constructive Identification -Overcoming the Curse of Dimensionality

The modelling of a nonlinear stochastic dynamical processes from data involves solving the problems of data gathering, preprocessing, model architecture selection, learning or adaptation, parametric evaluation and model validation. For a given model architecture such as associative memory networks, a common problem in non-linear modelling is the problem of the curse of dimensionality. A series of complementary data based constructive identification schemes, mainly based on but not limited to an operating point dependent fuzzy models, are introduced in this paper with the aim to overcome the curse of dimensionality. These include (i) a mixture of experts algorithm based on a forward constrained regression algorithm; (ii) an inherent parsimonious delaunay input space partition based piecewise local lineal modelling concept; (iii) a neurofuzzy model constructive approach based on forward orthogonal least squares and optimal experimental design and finally (iv) the neurofuzzy model construction algorithm based on basis functions that are Bézier Bernstein polynomial functions and the additive decomposition. Illustrative examples demonstrate their applicability, showing that the final major hurdle in data based modelling has almost been removed.

Forecasting Cyanobacterial Concentrations Using B-Spline Networks

Artificial neural networks have been used successfully in a number of areas of civil engineering, including hydrology and water resources engineering. In the vast majority of cases, multilayer perceptrons that are trained with the back-propagation algorithm are used. One of the major shortcomings of this approach is that it is difficult to elicit the knowledge about the input/output mapping that is stored in the trained networks. One way to overcome this problem is to use B-spline associative memory networks (AMNs), because their connection weights may be interpreted as a set of fuzzy membership functions and hence the relationship between the model inputs and outputs may be written as a set of fuzzy rules. In this paper, multilayer perceptrons and AMN models are compared, and their main advantages and disadvantages are discussed. The performance of both model types is compared in terms of prediction accuracy and model transparency for a particular water quality case study, the forecasting (4 weeks in adv...

A feedforward bidirectional associative memory

In contrast to conventional feedback bidirectional associative Memory (BAM) network models, a feedforward BAM network is developed based on a one-shot design algorithm of O(p(2)(n+m)) computational complexity, where p is the number of prototype pairs and n, m are the dimensions of the input/output bipolar vectors. The feedforward BAM is an n-p-m three-layer network of McCulloch-Pitts neurons with storage capacity 2(min{m,n}) and guaranteed perfect bidirectional recall. The overall network design procedure is fully scalable in the sense that any number p= or <2(min{m,n}) of bidirectional associations can be implemented. The prototype patterns may be arbitrarily correlated. With respect to inference performance, it is shown that the Hamming attractive radius of each prototype reaches the maximum possible value. Simulation studies and comparisons illustrate and support these theoretical developments.

Distributed synchrony in an attractor of spiking neurons

Recently, it has been shown that there exists a temporal synaptic learning curve, composed of potentiation for short time delays between pre- and post-synaptic neuronal spiking, and depression for spiking events occurring in the opposite order. We investigate the effect that such a curve can have on the temporal realization of an attractor in an associative memory network of spiking neurons. We find that an interesting result of this learning paradigm is the formation of distributed synchrony, i.e. spontaneous division of the Hebbian cell assembly into groups of cells that fire in a cyclic manner.

Periodic oscillatory solution of bidirectional associative memory networks with delays.

Periodic oscillatory phenomena of bidirectional associative memory (BAM) networks with axonal signal transmission delays are investigated by constructing suitable Lyapunov functionals and some analysis techniques. Some simple sufficient conditions are derived ensuring the existence and uniqueness of periodic oscillatory solutions of the BAM with delays, and all other solutions of the BAM converge exponentially to a periodic oscillatory solution. These conditions are presented in terms of system parameters, and have an important leading significance in the design and applications of periodic oscillatory neural circuits for the BAM with delays.

Associative memory in a multi-modular network.

Recent imaging studies suggest that object knowledge is stored in the brain as a distributed network of many cortical areas. Motivated by these observations, we study a multimodular associative memory network, whose functional goal is to store patterns with different coding levels--patterns that vary in the number of modules in which they are encoded. We show that in order to accomplish this task, synaptic inputs should be segregated into intramodular projections and intermodular projections, with the latter undergoing additional nonlinear dendritic processing. This segregation makes sense anatomically if the intermodular projections represent distal synaptic connections on apical dendrites. It is then straightforward to show that memories encoded in more modules are more resilient to focal afferent damage. Further hierarchical segregation of intermodular connections on the dendritic tree improves this resilience, allowing memory retrieval from input to just one of the modules in which it is encoded.

Associative database of protein sequences.

We present a new concept that combines data storage and data analysis in genome research, based on an associative network memory. As an illustration, 115 000 conserved regions from over 73 000 published sequences (i.e. from the entire annotated part of the SWISSPROT sequence database) were identified and clustered by a self-organizing network. Similarity and kinship, as well as degree of distance between the conserved protein segments, are visualized as neighborhood relationship on a two-dimensional topographical map. Such a display overcomes the restrictions of linear list processing and allows local and global sequence relationships to be studied visually. Families are memorized as prototype vectors of conserved regions. On a massive parallel machine, clustering and updating of the database take only a few seconds; a rapid analysis of incoming data such as protein sequences or ESTs is carried out on present-day workstations. Access to the database is available at http://www.bioinf.mdc-berlin.de/unter2.html++ + (hanke,lehmann,reich)@mdc-berlin.de; bork@embl-heidelberg.de

An indirect approach to adaptive control of nonlinear systems using quasi-ARMAX model

This paper presents an adaptive control scheme for nonlinear systems based on a quasi-ARMAX prediction model that is a specially constructed associative memory network consisting of multiple neurofuzzy models and is distinctive to usual neural networks in that it is linear in both the parameters to be estimated and the input variables to be synthesized in a control system. This advantage is taken to develop a nonlinear adaptive control scheme similarly to linear one.

The importance of nonlinear dendritic processing in multimodular memory networks

Recent imaging studies testify that objects are stored in the brain in a distributed network of discrete cortical areas. Motivated by these observations, we study a multi-modular associative memory network, whose functional goal is to store patterns of different coding levels, i.e., patterns that vary in the number of modules in which they are encoded. We show that in order to accomplish this task, the synaptic inputs to each neuron should be segregated to intramodular projections whose activations are summed up linearly, and inter-modular projections whose activations are summed up in a nonlinear fashion. This may reflect dendritic processing of distal synaptic connections. Moreover, we find that further hierarchical segregation of inter-modular connections on the dendritic tree improves memory retrieval from partial input cues, and improves resilience of the network to afferent damage.

Chaotic neural network for associative memory

Based on current research on applications of chaotic neuron network for information processing, the stability and convergence of chaotic neuron network are proved from the viewpoint of energy function. Moreover, a new auto-associative matrix is devised for artificial neural network composed of chaotic neurons, thus, an improved chaotic neuron network for associative memory is built up. Finally, the associative recalling process of the network is analyzed in detail and explanations of improvement are given.

Patterns and symbols treated in a three-layered associative memory network

This paper presents a new network for inference using patterns and symbols. Patterns are spatial distributions of feature vectors in nonlinear feature space made from original images. The new network is for acquiring concepts and inferring meanings of original images. Simulations using similar/different patterns are done to show the unique feature of the proposed network.

The Object Perceptron Learning Algorithm on Generalised Hopfield Networks for Associative Memory

We present a study of generalised Hopfield networks for associative memory. By analysing the radius of attraction of a stable state, the Object Perceptron Learning Algorithm (OPLA) and OPLA scheme are proposed to store a set of sample patterns (vectors) in a generalised Hopfield network with their radii of attraction as large as we require. OPLA modifies a set of weights and a threshold in a way similar to the perceptron learning algorithm. The simulation results show that the OPLA scheme is more effective for associative memory than both the sum-of-outer produce scheme with a Hopfield network and the weighted sum-of-outer product scheme with an asymmetric Hopfield network.

Robustness analysis of a class of discrete-time recurrent neural networks under perturbations

A robustness analysis is conducted for a large class of discrete-time recurrent neural networks for associative memories under perturbations of system parameters. The present paper aims to give an answer to the following question. Given a discrete-time neural network with specified stable memories (specified asymptotically stable equilibria), under what conditions will a perturbed model of the discrete-time neural network possess stable memories that are close (in distance) to the stable memories of the unperturbed discrete-time neural network model? Robustness stability results for perturbed discrete-time neural network models are established and conditions are obtained for the existence of asymptotically stable equilibria of the perturbed discrete-time neural network models which are near the asymptotically stable equilibria of the original unperturbed neural networks. In the present results, quantitative estimates (explicit estimates of bounds) are given for the distance between the corresponding equilibrium points of the unperturbed and perturbed discrete-time neural network models considered herein.

Dual-mode space-varying self-designing cellular neural networks for associative memory

A dual-mode space-varying CNN is proposed for associative memory. In the learning mode the CNN is used as a designer network which computes the weights to be used in the recall mode. Learning involves only local information, i.e., available inside each cell without extra interconnections, It allows to us exploit the analog and parallel computational power of the CNN chip, not only for information storage and retrieval, but also for the design of the CNN itself. Simulation results on the capacity obtained by the proposed learning algorithm are presented.