Associative Memory Networks Research Articles

Design and Analysis of Quaternion-Valued Neural Networks for Associative Memories

This paper addresses the problem of designing associative memories based on quaternion-valued neural networks (QVNNs). A system designing procedure for QVNNs is developed by employing quaternion matrix decomposition, and a given set of states can be assigned as the equilibrium points of the designed QVNNs. Moreover, some sufficient conditions for the asymptotic stability of the equilibrium points are obtained via Lyapunov’s direct method. Numerical simulations manifest that the constructed QVNNs work efficiently on storing and retrieving blurred gray-scale and true color images.

On simulating one-trial learning using morphological neural networks

“Learning once, remembering forever”, this wonderful cognitive phenomenon sometimes occurs in the learning process of human beings. Psychologists call this psychological phenomenon “one-trial learning”. The traditional artificial neural networks can simulate the psychological phenomenon of “implicit learning”, but can’t simulate the cognitive phenomenon of “one-trial learning”. Therefore, cognitive psychology gives a challenge to the traditional artificial neural networks. From two aspects of theory and practice in this paper, the possibility of simulating this kind of psychological phenomenon was explored by using morphological neural networks. This paper takes advantage of morphological associative memory networks to realize the simulation of “one-trial learning” for the first time, and gives 5 simulating practical examples. Theoretical analysis and simulation experiments show that the morphological associative memory networks are a higher effective machine learning method, and can better simulate the cognitive phenomenon of “one-trial learning”, therefore provide a theoretical basis and technological support for the study of intelligent science and cognitive science.

A Critical Role for the Nucleus Reuniens in Long-Term, But Not Short-Term Associative Recognition Memory Formation.

Recognition memory for single items requires the perirhinal cortex (PRH), whereas recognition of an item and its associated location requires a functional interaction among the PRH, hippocampus (HPC), and medial prefrontal cortex (mPFC). Although the precise mechanisms through which these interactions are effected are unknown, the nucleus reuniens (NRe) has bidirectional connections with each regions and thus may play a role in recognition memory. Here we investigated, in male rats, whether specific manipulations of NRe function affected performance of recognition memory for single items, object location, or object-in-place associations. Permanent lesions in the NRe significantly impaired long-term, but not short-term, object-in-place associative recognition memory, whereas single item recognition memory and object location memory were unaffected. Temporary inactivation of the NRe during distinct phases of the object-in-place task revealed its importance in both the encoding and retrieval stages of long-term associative recognition memory. Infusions of specific receptor antagonists showed that encoding was dependent on muscarinic and nicotinic cholinergic neurotransmission, whereas NMDA receptor neurotransmission was not required. Finally, we found that long-term object-in-place memory required protein synthesis within the NRe. These data reveal a specific role for the NRe in long-term associative recognition memory through its interactions with the HPC and mPFC, but not the PRH. The delay-dependent involvement of the NRe suggests that it is not a simple relay station between brain regions, but, rather, during high mnemonic demand, facilitates interactions between the mPFC and HPC, a process that requires both cholinergic neurotransmission and protein synthesis.SIGNIFICANCE STATEMENT Recognizing an object and its associated location, which is fundamental to our everyday memory, requires specific hippocampal-cortical interactions, potentially facilitated by the nucleus reuniens (NRe) of the thalamus. However, the role of the NRe itself in associative recognition memory is unknown. Here, we reveal the crucial role of the NRe in encoding and retrieval of long-term object-in-place memory, but not for remembrance of an individual object or individual location and such involvement is cholinergic receptor and protein synthesis dependent. This is the first demonstration that the NRe is a key node within an associative recognition memory network and is not just a simple relay for information within the network. Rather, we argue, the NRe actively modulates information processing during long-term associative memory formation.

Diesel Engine Combustion Control Based on Cerebellar Model Articulation Controller (CMAC) in Feedback Error Learning

The trend started in 1997 with the introduction of common rail injection and after some protocols set new targets for overall CO2 emissions. As the diesel engine emits less CO2 than its gasoline counterpart, it kept conquering more and more market shares. Conventional diesel engine control design is mainly based on the maps techniques which required too much time, money and human resources under the number of experiments under various environmental conditions. This makes increasing system complexity. In our authors group, we have proposed that the control structure has the feedback error learning, two-degree-of-freedom controller configuration, with advanced neural networks (NNs) as the feedforward controller along the model-based control method. On the other hand, a cerebellar model articulation controller (CMAC) is a non-fully connected perceptron like associative memory network with overlapping receptive fields, which is used to resolve problems that involve rapid growth and the learning difficulty. Then CMACs have the advantages of good generalization capability, fast learning ability, and simple computation. To our best knowledge, this is new introduce the cerebellar model articulation controller (CMAC) for the control diesel engine combustion control. The effectiveness of the proposed method will be confirmed through numerical simulations based on the Tokyo University diesel engine model with triple fuel injections.

Investigating feedback effects in the field of brand extension using brand concept maps

PurposeThe purpose of this paper is to apply the (advanced) brand concept maps (BCM) approach to reinvestigate previous findings on feedback effects resulting from brand extension information (BEI) and to explore whether this information affects the structure of a brand’s associative network.Design/methodology/approachThis research builds on the associative network memory model, as well as Keller’s conceptualization of customer-based brand equity, and uses a series of empirical studies with a total of 839 respondents in two different countries.FindingsThe findings reveal that BEI has no significant impact on the structure of the parent brand’s associative network at the individual level. Furthermore, key brand image dimensions (i.e. favorability, strength, and uniqueness of brand associations) are not affected.Research limitations/implicationsBy applying the (advanced) BCM approach, this paper is able to address shortcomings that are incorporated with the use of Likert scales for measuring a brand’s image and for investigating feedback effects in the field of brand extension. As the results indicate that the identification of feedback effects might be influenced by the approach used to measure a brand’s image, this paper calls for further investigations of feedback effects on a brand’s image.Originality/valueData from three empirical studies provide insights into the cognitive processing of BEI and their impact on a brand’s associative network.

Multilayer feed forward neural networks for non-linear continuous bidirectional associative memory

The study of Bidirectional associative memory (BAM), with recurrent neural networks and symmetric as well as asymmetric weights, has already been undertaken in various different ways. Using two phases of learning for multilayer neural network architecture in the present paper, a multilayer feed forward neural network model has been proposed to construct the non-linear continuous BAM for pattern association. In the first phase an input pattern is presented to input layer and back propagation learning rule is used to train the network in the feed forward direction for the corresponding associated output pattern. In second phase the output pattern is presented to output layer as input and again the back propagation learning rule is used to train the same network in feedback direction for the corresponding associated input pattern. In these two passes i.e. forward pass and backward pass, the interconnection weights are considered asymmetric. This training process continues till the network does not converge to the final optimal weights by minimizing the mean square errors in both the directions simultaneously. At this convergence of weights the input and output layers exhibit the stability and the performance of such type of BAM is evaluated for the test pattern set while the simulation results exhibit the better performance of associative memory for the proposed method. The storage capacity of network is also increased due to the non-linear mapping between input-output pattern pairs and also the occurrence of spurious states reduces during the recalling process.

Neural Networks Retrieving Boolean Patterns in a Sea of Gaussian Ones

Restricted Boltzmann Machines are key tools in Machine Learning and are described by the energy function of bipartite spin-glasses. From a statistical mechanical perspective, they share the same Gibbs measure of Hopfield networks for associative memory. In this equivalence, weights in the former play as patterns in the latter. As Boltzmann machines usually require real weights to be trained with gradient descent like methods, while Hopfield networks typically store binary patterns to be able to retrieve, the investigation of a mixed Hebbian network, equipped with both real (e.g., Gaussian) and discrete (e.g., Boolean) patterns naturally arises. We prove that, in the challenging regime of a high storage of real patterns, where retrieval is forbidden, an extra load of Boolean patterns can still be retrieved, as long as the ratio among the overall load and the network size does not exceed a critical threshold, that turns out to be the same of the standard Amit-Gutfreund-Sompolinsky theory. Assuming replica symmetry, we study the case of a low load of Boolean patterns combining the stochastic stability and Hamilton-Jacobi interpolating techniques. The result can be extended to the high load by a non rigorous but standard replica computation argument.

Design and analysis of associative memories based on external inputs of continuous bidirectional associative networks

This paper presents an extended continuous bidirectional associative memory network (CBAM) and a new ring recurrent network to behave as associative memories with external inputs. The proposed networks are robust in terms of design parameter selection. Some globally exponential stable criteria are derived for the networks with high storage capacity. The approach, by generating networks where the input data are fed via external inputs rather than initial conditions, enables multiple prototype patterns to be retrieved simultaneously. The results improve and extend some previous related works. Recurring to the numerical method, several applicable examples are given to illustrate the effectiveness of the proposed networks.

Automatic activation of alcohol cues by child maltreatment related words: a replication attempt in a different treatment setting

BackgroundA growing body of research attempts to clarify the underlying mechanisms of the association between emotional maltreatment and alcohol dependence (AD). In a preceding study, we found considerable support for a specific priming effect in subjects with AD and emotional abuse experiences receiving alcohol rehabilitation treatment. We concluded that maltreatment related cues can automatically activate an associative memory network comprising cues eliciting craving as well as alcohol-related responses. Generalizability of the results to other treatment settings remains unclear because of considerable differences in German treatment settings as well as insufficiently clarified influences of selection effects. As replication studies in other settings are necessary, the current study aimed to replicate the specific priming effect in a qualified detoxification sample.Results22 AD subjects (n = 10 with emotional abuse vs. n = 12 without emotional abuse) participated in a priming experiment. Comparison data from 34 healthy control subjects were derived from the prior study. Contrary to our hypothesis, we did not find a specific priming effect.ConclusionsWe could not replicate the result of an automatic network activation by maltreatment related words in a sample of subjects with AD and emotional abuse experiences receiving qualified detoxification treatment. This discrepancy might be attributed to reasons related to treatment settings as well as to methodological limitations. Future work is required to determine the generalizability of the specific priming effect before valid conclusions regarding automatic activation can be drawn.

Dynamic functional connectivity shapes individual differences in associative learning.

Current neuroscientific research has shown that the brain reconfigures its functional interactions at multiple timescales. Here, we sought to link transient changes in functional brain networks to individual differences in behavioral and cognitive performance by using an active learning paradigm. Participants learned associations between pairs of unrelated visual stimuli by using feedback. Interindividual behavioral variability was quantified with a learning rate measure. By using a multivariate statistical framework (partial least squares), we identified patterns of network organization across multiple temporal scales (within a trial, millisecond; across a learning session, minute) and linked these to the rate of change in behavioral performance (fast and slow). Results indicated that posterior network connectivity was present early in the trial for fast, and later in the trial for slow performers. In contrast, connectivity in an associative memory network (frontal, striatal, and medial temporal regions) occurred later in the trial for fast, and earlier for slow performers. Time-dependent changes in the posterior network were correlated with visual/spatial scores obtained from independent neuropsychological assessments, with fast learners performing better on visual/spatial subtests. No relationship was found between functional connectivity dynamics in the memory network and visual/spatial test scores indicative of cognitive skill. By using a comprehensive set of measures (behavioral, cognitive, and neurophysiological), we report that individual variations in learning-related performance change are supported by differences in cognitive ability and time-sensitive connectivity in functional neural networks. Hum Brain Mapp 37:3911-3928, 2016. © 2016 Wiley Periodicals, Inc.

Topological Gaussian ARAM for biologically inspired topological map building

This paper presents a new neural network for online topological map building inspired by beta oscillations and hippocampal place cell learning. The memory layer represents the hippocampus, the input layer represents the entorhinal, and the $$\rho$$ is the orientation system. In this model, multiple-scale entorhinal grid cell activations form the input layer feature patterns, which are categorized by hippocampal place cells (nodes) and act as spatial categories in the memory layer. Top-down attentive matching and mismatch-mediated reset (beta oscillations), which are triggered by the orientation system, overcome the stability-plasticity dilemma and prevent the catastrophic forgetting of place cell maps. In our proposed method, nodes in the topological map represent place cells (robot location), while edges connect nodes and store robot action (i.e., orientation, direction). Our method is based upon a multi-channel Adaptive Resonance Associative Memory (ARAM) network architecture to obtain multiple sensory sources for topological map building. It comprises two layers: input and memory. The input layer collects sensory data and incrementally clusters the obtained information into a set of topological nodes. In the memory layer, the clustered information is used as a topological map where nodes are associated with actions. The advantages of the proposed method are: (1) it does not require high-level cognitive processes and prior knowledge to make it work in a natural environment; and (2) it can process multiple sensory sources simultaneously in continuous space, which is crucial for real-world robot navigation. Thus, we combine our Topological Gaussian ARAM method (TGARAM) with incremental principle component analysis to constitute a basis for topological map building. Lastly, the proposed method was validated using several standardized benchmark datasets.

Clustering of neural code words revealed by a first-order phase transition.

A network of neurons in the central nervous system collectively represents information by its spiking activity states. Typically observed states, i.e., code words, occupy only a limited portion of the state space due to constraints imposed by network interactions. Geometrical organization of code words in the state space, critical for neural information processing, is poorly understood due to its high dimensionality. Here, we explore the organization of neural code words using retinal data by computing the entropy of code words as a function of Hamming distance from a particular reference codeword. Specifically, we report that the retinal code words in the state space are divided into multiple distinct clusters separated by entropy-gaps, and that this structure is shared with well-known associative memory networks in a recallable phase. Our analysis also elucidates a special nature of the all-silent state. The all-silent state is surrounded by the densest cluster of code words and located within a reachable distance from most code words. This code-word space structure quantitatively predicts typical deviation of a state-trajectory from its initial state. Altogether, our findings reveal a non-trivial heterogeneous structure of the code-word space that shapes information representation in a biological network.

Place Cell Networks in Pre-weanling Rats Show Associative Memory Properties from the Onset of Exploratory Behavior.

Place cells are hippocampal pyramidal cells that are active when an animal visits a restricted area of the environment, and collectively their activity constitutes a neural representation of space. Place cell populations in the adult rat hippocampus display fundamental properties consistent with an associative memory network: the ability to 1) generate new and distinct spatial firing patterns when encountering novel spatial contexts or changes in sensory input (“remapping”) and 2) reinstate previously stored firing patterns when encountering a familiar context, including on the basis of an incomplete/degraded set of sensory cues (“pattern completion”). To date, it is unknown when these spatial memory responses emerge during brain development. Here, we show that, from the age of first exploration (postnatal day 16) onwards, place cell populations already exhibit these key features: they generate new representations upon exposure to a novel context and can reactivate familiar representations on the basis of an incomplete set of sensory cues. These results demonstrate that, as early as exploratory behaviors emerge, and despite the absence of an adult-like grid cell network, the developing hippocampus processes incoming sensory information as an associative memory network.

Control Strategy Based on Associative Memory Networks for a Grid-Side Converter in On-Grid Renewable Generation Systems Under Generalized Unbalanced Grid Voltage Conditions

This paper presents a control strategy based on Associative Memory Networks (AMN) for controlling a grid-side converter in the On-grid renewable generation systems under generalized unbalanced grid voltage conditions. In the proposed scheme, the controller operates in a rotating synchronous reference frame, where standard PI controllers are utilized in the fast inner dual current loops to track the current references, while in the outer loop, the AMN is used as a nonlinear adaptive integrator combined with a simple proportional controller for the DC link voltage regulation. By using the simulation study, it is shown that the overshoot of the proposed control system in transient state is lower compared with that of the optimal PI controller. Consequently the overvoltage of the DC link capacitor due to abrupt changes of the active power generated by renewable generation systems under both balanced and unbalanced grid voltage conditions could be effectively avoided.

Gamma EEG Activation During Picture‐Word Semantic Processing in 3 Year Olds Varies As A Function of Gender And Infant Diet

Semanticrelationships between images and words essential for language comprehension andcognitive function begin to emerge in toddlers, but it is not known if theearly development of these relationships is modulated by infant diet andgender. In this study 254 healthy 3yr‐olds [breast‐fed (BF: 86, 42 boys), or fed milk formula (MF: 88, 40 boys),or soy formula (SF: 80, 44 boys) as infants] viewed images of known objects/animalswhile hearing single words that were either congruent or incongruent with theimages, or were non‐words. For each ofthese image‐word conditions we used ANOVA to compare group, gender, and groupby gender‐related changes in frontal and temporal gamma (30–50 Hz) EEG powerduring the word relative to an immediately preceding baseline. Gamma activityis considered to index activation of associative memory networks, with gammaresponse amplitude increasing with memory strength. All word types elicitedincreases in gamma amplitude, and across groups at both frontal and temporalsites increases were significantly greater to congruent and incongruent wordscompared with non‐words (p < 0.05). Significant gender influences were evident as main effects andgroup‐gender interactions (p < 0.05), reflecting greater gamma amplitudeincreases in boys than girls—most prominent in SF children and least evident inBF children—and in SF compared with MF boys. These findings are the first tolink infant diet and gender to brain activity indexing cross‐modality semanticrelationships in toddlers. The results provide evidence that early infantdiet‐gender interactions may contribute differentially to the development offunctional neural networks involved in semantic processing in children, andsuggest that semantic relationships at this time are more strongly developed inSF boys.Support or Funding InformationFunded by USDA‐ARS Project 6026‐51000‐010‐05S

A Self-Organizing Incremental Spatiotemporal Associative Memory Networks Model for Problems with Hidden State.

Identifying the hidden state is important for solving problems with hidden state. We prove any deterministic partially observable Markov decision processes (POMDP) can be represented by a minimal, looping hidden state transition model and propose a heuristic state transition model constructing algorithm. A new spatiotemporal associative memory network (STAMN) is proposed to realize the minimal, looping hidden state transition model. STAMN utilizes the neuroactivity decay to realize the short-term memory, connection weights between different nodes to represent long-term memory, presynaptic potentials, and synchronized activation mechanism to complete identifying and recalling simultaneously. Finally, we give the empirical illustrations of the STAMN and compare the performance of the STAMN model with that of other methods.

Handwritten Javanese Character Recognition Using Several Artificial Neural Network Methods

Javanese characters are traditional characters that are used to write the Javanese language. The Javanese language is a language used by many people on the island of Java, Indonesia. The use of Javanese characters is diminishing more and more because of the difficulty of studying the Javanese characters themselves. The Javanese character set consists of basic characters, numbers, complementary characters, and so on. In this research we have developed a system to recognize Javanese characters. Input for the system is a digital image containing several handwritten Javanese characters. Preprocessing and segmentation are performed on the input image to get each character. For each character, feature extraction is done using the ICZ-ZCZ method. The output from feature extraction will become input for an artificial neural network. We used several artificial neural networks, namely a bidirectional associative memory network, a counterpropagation network, an evolutionary network, a backpropagation network, and a backpropagation network combined with chi2. From the experimental results it can be seen that the combination of chi2 and backpropagation achieved better recognition accuracy than the other methods.

A memristive Hopfield network for associative memory

A memristive Hopfield network for associative memory

Dynamics in Four-Neuron Bidirectional Associative Memory Networks with Inertia and Multiple Delays

Bidirectional associative memory (BAM) networks play an important role in various fields such as optimization, pattern recognition, classification, signal and image processing, parallel computation and associative memory. In this paper, four-neuron BAM networks with inertia and multiple delays are considered. By analyzing the distribution of the eigenvalues of the associated characteristic transcendental equation, local stability criteria are obtained for various system parameters and time delays. By choosing the sum of time delays as a bifurcation parameter, we found that Hopf bifurcation occurs when the sum of time delays passes through a sequence of critical values. Some explicit formulae for determining the stability and the direction of the Hopf bifurcation periodic solutions are obtained by using the normal form theory and center manifold theory. Some numerical simulations are carried out to support theoretical predictions. Our results are new and supplement some previously known studies.

Memristor-based Willshaw network: Capacity and robustness to noise in the presence of defects

The recent realization of memristors, nanodevices remarkably similar to biological synapses, opened the possibility to fabricate highly scalable artificial neural networks. While the physical implementation of such networks is still emerging, it is useful to perform simulations to determine the impact of non-ideal devices or device faults in the performance of memory networks. Here, we numerically evaluate a memristor-based Willshaw associative memory network, studying its capacity and robustness to noise as a function of defects probability and device parameter variations. Two types of defective memristors are addressed (stuck-at-0 and stuck-at-1) and Gaussian distributions are imposed to their threshold voltages, ON and OFF resistances. We conclude that the type and number of defects strongly determine how the network should be operated. The reading current threshold also plays a key role in determining the network's capacity and robustness to noise. Furthermore, there is a maximum defect percentage above which the network can no longer reliably store information. We also found that the memristor-based Willshaw network is more sensitive to resistance variance than to threshold voltage variance.