Hetero-associative Network Research Articles

A Neural Network Model of the Entorhinal Cortex and Hippocampus for Event-Order Memory Processing

To solve a navigation task based on experiences, we need a mechanism to associate places with objects and recall them along the course of action. In a reward-oriented task, if the route to a reward location is simulated in mind after experiencing it once, it might be possible that the reward is gained efficiently. One way to solve this is to incorporate a biologically plausible mechanism. In this study, we propose a neural network that stores a sequence of events associated with a reward. The proposed network recalls the reward location by tracing them in its mind in order. We simulated a virtual mouse that explores a figure-eight maze and recalls the route to the reward location. During the learning period, a sequence of events related to firing along a passage was temporarily stored in the heteroassociative network, and the sequence of events is consolidated in the synaptic weight matrix when a reward is fed. For retrieval, an impetus input internally generates the sequential activation of conjunctive cue–place cells toward the reward location. In the figure-eight maze task, the location of the reward was estimated by mind travel, irrespective of whether the reward is in the counterclockwise or distant clockwise route. The mechanism of efficiently reaching the goal by mind travel in the brain based on experiences is beneficial for mobile service robots that perform autonomous navigation.

Bidirectional Hetero-Associative Memory Network With Flexible Sensors and Cloud Computing for Blood Leakage Detection in Intravenous and Dialysis Therapy

Infiltration and blood leakage are serious life-threatening complications during intravenous administration and dialysis therapy. Serious infiltrations, such as drips and drugs, may cause inflection, sepsis, and extent of tissue damage. Blood leakage and blood loss are serious complications, as it takes only a few minutes to lose over 40% of adult blood volume, potentially resulting in mortality. Therefore, we propose integrating the flexible sensors and bidirectional hetero-associative memory (BHAM) network into a cloud computing-based warning tool for the detection of infiltration and blood leakage. The flexible sensors are arranged in two array patterns for liquid leakage detection and are fabricated on a soft substrate via a screen-printing technique. The BHAM network constructs a virtual alarm unit in an embedded system or tablet PC. This early warning unit can be employed to identify risk levels via a wireless network and cloud computing, and then indicates warning information on a remote monitor device or a mobile device. The proposed detection model has been validated experimentally, and the experimental results demonstrate its feasibility.

Heteroassociative storage of hippocampal pattern sequences in the CA3 subregion.

BackgroundRecent research suggests that the CA3 subregion of the hippocampus has properties of both autoassociative network, due to its ability to complete partial cues, tolerate noise, and store associations between memories, and heteroassociative one, due to its ability to store and retrieve sequences of patterns. Although there are several computational models of the CA3 as an autoassociative network, more detailed evaluations of its heteroassociative properties are missing.MethodsWe developed a model of the CA3 subregion containing 10,000 integrate-and-fire neurons with both recurrent excitatory and inhibitory connections, and which exhibits coupled oscillations in the gamma and theta ranges. We stored thousands of pattern sequences using a heteroassociative learning rule with competitive synaptic scaling.ResultsWe showed that a purely heteroassociative network model can (i) retrieve pattern sequences from partial cues with external noise and incomplete connectivity, (ii) achieve homeostasis regarding the number of connections per neuron when many patterns are stored when using synaptic scaling, (iii) continuously update the set of retrievable patterns, guaranteeing that the last stored patterns can be retrieved and older ones can be forgotten.DiscussionHeteroassociative networks with synaptic scaling rules seem sufficient to achieve many desirable features regarding connectivity homeostasis, pattern sequence retrieval, noise tolerance and updating of the set of retrievable patterns.

Optimal pair of hippocampal CA1 phase response curve and spike-timing-dependent plasticity for hetero-associative memory

Recently reported experimental findings suggest that the hippocampal CA1 network stores spatio-temporal spike patterns and retrieves temporally reversed [1] and spread-out [2] patterns. In this paper, we explore the idea that the properties of the neural interactions and the synaptic plasticity rule in the CA1 network enable it to function as a hetero-associative memory recalling such reversed and spread-out spike patterns. In line with Lengyel's speculation [3], we derive optimally designed spike-timing-dependent plasticity (STDP) rules that are matched to neural interactions formalized in terms of phase response curves (PRCs) for performing the hetero-associative memory function (see Figure ​Figure1).1). First, we formulate a hetero-associative memory network recalling not only the normal spike patterns, but also the reversed and doubly spread-out patterns as a phase oscillator model consisting of an STDP and a PRC. Next, we analytically derive the mutual information between a stored phase pattern and a network output for evaluating memory retrieval performance. By maximizing an object function given by the mutual information, we search for STDP window functions that are optimal for retrieval of normal and doubly spread-out patterns under the constraint that the PRCs are those of CA1 pyramidal neurons recorded in vitro [4]. Figure 1 Outline of our approach to derive pairs of PRCs and STDPs optimally recalling normal, reversed, and doubly spread-out patterns.

Dual Coding with STDP in a Spiking Recurrent Neural Network Model of the Hippocampus

The firing rate of single neurons in the mammalian hippocampus has been demonstrated to encode for a range of spatial and non-spatial stimuli. It has also been demonstrated that phase of firing, with respect to the theta oscillation that dominates the hippocampal EEG during stereotype learning behaviour, correlates with an animal's spatial location. These findings have led to the hypothesis that the hippocampus operates using a dual (rate and temporal) coding system. To investigate the phenomenon of dual coding in the hippocampus, we examine a spiking recurrent network model with theta coded neural dynamics and an STDP rule that mediates rate-coded Hebbian learning when pre- and post-synaptic firing is stochastic. We demonstrate that this plasticity rule can generate both symmetric and asymmetric connections between neurons that fire at concurrent or successive theta phase, respectively, and subsequently produce both pattern completion and sequence prediction from partial cues. This unifies previously disparate auto- and hetero-associative network models of hippocampal function and provides them with a firmer basis in modern neurobiology. Furthermore, the encoding and reactivation of activity in mutually exciting Hebbian cell assemblies demonstrated here is believed to represent a fundamental mechanism of cognitive processing in the brain.

Pattern Recognition by High-Capacity Polynomial Bidirectional Hetero-Associative Network

This investigation presents a novel method of image processing using the polynomial bidirectional hetero-associative network (PBHAN). This network can be used for industrial application of optical character recognition. According to the results of detailed simulations, the PBHAN has a higher capacity for pattern pair storage than do the conventional bidirectional associative memories and fuzzy memories. The practical capacity of a PBHAN considering fault tolerance is discussed. The fault tolerance requirement leads to the discovery of the attraction radius of the basin for each stored pattern pair. PBHAN takes advantage of fuzzy characteristics in evolution equations such that the signal-noise-ratio is significantly increased. In this work, we apply the result of this research to pattern recognition problems. The practical capacity of fuzzy data recognition using PBHAN and considering fault tolerance in the worst case is also estimated. Simulation results are presented to verify the derived theory.

Animat navigation using a cognitive graph.

This article describes a computational model of the hippocampus that makes it possible for a simulated rat to navigate in a continuous environment containing obstacles. This model views the hippocampus as a "cognitive graph", that is, a hetero-associative network that learns temporal sequences of visited places and stores a topological representation of the environment. Calling upon place cells, head direction cells, and "goal cells", it suggests a biologically plausible way of exploiting such a spatial representation for navigation that does not require complicated graph-search algorithms. Moreover, it permits "latent learning" during exploration, that is, the building of a spatial representation without the need of any reinforcement. When the rat occasionally discovers some rewarding place it may wish to rejoin subsequently, it simply records within its cognitive graph, through a series of goal and sub-goal cells, the direction in which to move from any given start place. Accordingly, the model implements a simple "place-recognition-triggered response" navigation strategy. Two implementations of place cell management are studied in parallel. The first one associates place cells with place fields that are given a priori and that are uniformly distributed in the environment. The second one dynamically recruits place cells as exploration proceeds and adjusts the density of such cells to the local complexity of the environment. Both implementations lead to identical results. The article ends with a few predictions about results to be expected in experiments involving simultaneous recordings of multiple cells in the rat hippocampus.

Fuzzy data processing using polynomial bidirectional hetero-associative network

This investigation presents a novel method of fuzzy data processing using polynomial bidirectional hetero-associative network (PBHAN). This has a higher capacity for pattern pair storage than that of the conventional bidirectional associative memories (BAMs) and fuzzy memories. In addition, a new energy function is defined. The PBHAN takes advantage of fuzzy characteristics in evolution equations such that the signal-noise-ratio (SNR) is significantly increased. The energy of the PBHAN defined by the proposed energy function decreases as the recall process proceeds, thereby ensuring the stability of the system. In this work, we prove the stability of fuzzy data processing using PBHAN. The increase of SNR consequently enhances the capacity of the PBHAN. The capacity of the fuzzy data processing using PBHAN in the worst case is also estimated.

Dynamics of selective recall in an associative memory model with one-to-many associations

The dynamics of selective recall in an associative memory model are analyzed in the scenario of one-to-many association. One-to-many association is one of the most important characteristics of our memory system because a homophone, for example, associates with more than one word and each word can have several meanings. The present model, which can deal with one-to-many association, consists of a heteroassociative network and an autoassociative network. In the heteroassociative network, a mixture of associative items in one-to-many association is recalled by a key item. In the autoassociative network, the selective recall of one of the associative items is examined by providing a seed of a target item either to the heteroassociative network (Model 1) or to the autoassociative network (Model 2). We show by both simulation studies and theoretical analysis that the critical similarity of Model 2 is not sensitive to the change in the dimension ratio of key vectors to associative vectors, and it has smaller critical similarity (correlation between the seed and the target item) than Model 1 for a large initial overlap. On the other hand, we show that Model 1 has smaller critical similarity for a small initial overlap. We also show that unreachable equilibrium states exist in the proposed model. There is a critical loading rate alphar where the reachable equilibrium states are disappeared. Above the critical loading rate alphar, which is smaller than the storage capacity alphac, all equilibrium states are stable, but cannot be reached.

Self-organizing algorithms for generalized eigen-decomposition

We discuss a new approach to self-organization that leads to novel adaptive algorithms for generalized eigen-decomposition and its variance for a single-layer linear feedforward neural network. First, we derive two novel iterative algorithms for linear discriminant analysis (LDA) and generalized eigen-decomposition by utilizing a constrained least-mean-squared classification error cost function, and the framework of a two-layer linear heteroassociative network performing a one-of-m classification. By using the concept of deflation, we are able to find sequential versions of these algorithms which extract the LDA components and generalized eigenvectors in a decreasing order of significance. Next, two new adaptive algorithms are described to compute the principal generalized eigenvectors of two matrices (as well as LDA) from two sequences of random matrices. We give a rigorous convergence analysis of our adaptive algorithms by using stochastic approximation theory, and prove that our algorithms converge with probability one.

Automatic Setting of Article Format Through Neural Networks

The automatic format-setting of journal articles for reducing the workload of computer users involves two processes: automatic acquisition of article format and automatic recall of article formal. Several neural networks have been explored to implement the two processes. The advantages and disadvantages of these neural networks are evaluated in comparison with capabilities of conventional computer programs. A heteroassociative back-propagation network has been developed for the automatic acquisition process. This network excels over computer programs because of its abilities in learning and generalizing implicit knowledge from examples. A bidirectional associative memory network, a Boltzman network, and an autoassociative back-propagation network have been investigated for the automatic recall process. None of them excel over computer programs in terms of recall accuracy.

Storage capacity analysis of a model of human associative processing (HASP)

This paper analyzes the storage capacity of the associative memory network (HASP). HASP is an associative memory network, which is composed of the heteroassociative memory network using the correlation matrix, and the mutually inhibitory network. The feature of the network is that the one-to-many association can be realized, which has been difficult in the networks hitherto proposed. Because a mutually inhibitory network can effectively eliminate the crosstalk noise produced in the heteroassociative memory network, it is expected that the storage capacity will be greater compared to the previously proposed networks. Willshaw et al. analyzed the storage capacity in the heteroassociative memory [4]. In the present study, a similar method is used to analyze and evaluate theoretically the storage capacity of HASP. As a result, it is shown that HASP can realize twice or larger storage capacity compared to the heteroassociative memory networks, when the keys and the associative vectors are sparsely encoded. It is verified that the result of the analysis agrees well with the result of the computer simulations.

Theta/gamma networks with slow NMDA channels learn sequences and encode episodic memory: role of NMDA channels in recall.

This paper examines the role of slow N-methyl-D-aspartate (NMDA) channels (deactivation approximately 150 msec) in networks that multiplex different memories in different gamma subcycles of a low frequency theta oscillation. The NMDA channels are in the synapses of recurrent collaterals and govern synaptic modification in accord with known physiological properties. Because slow NMDA channels have a time constant that spans several gamma cycles, synaptic connections will form between cells that represent different memories. This enables brain structures that have slow NMDA channels to store heteroassociative sequence information in long-term memory. Recall of this stored sequence information can be initiated by presentation of initial elements of the sequence. The remaining sequence is then recalled at a rate of one memory every gamma cycle. A new role for the NMDA channel suggested by our finding is that recall at gamma frequency works well if slow NMDA channels provide the dominant component of the EPSP at the synapse of recurrent collaterals: The slow onset of these channels and their long duration allows the firing of one memory during one gamma cycle to trigger the next memory during the subsequent gamma cycle. An interesting feature of the readout mechanism is that the activation of a given memory is due to cumulative input from multiple previous memories in the stored sequence, not just the previous one. The network thus stores sequence information in a doubly redundant way: Activation of a memory depends on the strength of synaptic inputs from multiple cells of multiple previous memories. The cumulative property of sequence storage has support from the psychophysical literature. Cumulative learning also provides a solution to the disambiguation problem that occurs when different sequences have a region of overlap. In a final set of simulations, we show how coupling an autoassociative network to a heteroassociative network allows the storage of episodic memories (a unique sequence of briefly occurring known items). The autoassociative network (cortex) captures the sequence in short-term memory and provides the accurate, time-compressed repetition required to drive synaptic modification in the heteroassociative network (hippocampus). This is the first mechanistically detailed model showing how known brain properties, including network oscillations, recurrent collaterals, AMPA channels, NMDA channel subtypes, the ADP, and the AHP can act together to accomplish memory storage and recall.