Associative Memory Model Research Articles

Información, estabilidad y complejidad de aprendizaje en memorias asociativas

espanolEn este trabajo se estudia la relacion entre la canti-dad de informacion y la complejidad de aprendizaje en el modelo discreto de Hopfield de memoria asociativa. Mas especificamente, se analiza, desde un punto de vista estadistico, la relacion entre la probabilidad de que un estado del sistema sea un equilibrio estable (memoria) y un valor de entropia o incertidumbre asociado a el. Se realizan experimentos computacionales para corroborar estos resultados. EnglishThe relationship between amount of information and learning complexity is studied for the discrete Hopfield model of associative memory. More precisely, we analize, from a statistical point of view, the relation between the probability of a state of the system to be a stable equilib-rium (i.e. a memory) and a value of entropy or uncertainty associated to it. Computer experiments are made to confirm these results.

Properties of Coupled Oscillator Model for Bidirectional Associative Memory

In this study, we consider the stationary state and dynamical properties of a coupled oscillator model for bidirectional associative memory. For the stationary state, we apply the replica method to obtain self-consistent order parameter equations. The theoretical results for the storage capacity and overlap agree well with the numerical simulation. For the retrieval process, we apply statistical neurodynamics to include temporal noise correlations. For the successful retrieval process, the theoretical result obtained with the fourth-order approximation qualitatively agrees with the numerical simulation. However, for the unsuccessful retrieval process, higher-order noise correlations suppress severely; therefore, the maximum value of the overlap and the relaxation time are smaller than those of the numerical simulation. The reasons for the discrepancies between the theoretical result and numerical simulation, and the validity of our analysis are discussed.

Long-Term Memory in Drosophila Is Influenced by Histone Deacetylase HDAC4 Interacting with SUMO-Conjugating Enzyme Ubc9.

HDAC4 is a potent memory repressor with overexpression of wild type or a nuclear-restricted mutant resulting in memory deficits. Interestingly, reduction of HDAC4 also impairs memory via an as yet unknown mechanism. Although histone deacetylase family members are important mediators of epigenetic mechanisms in neurons, HDAC4 is predominantly cytoplasmic in the brain and there is increasing evidence for interactions with nonhistone proteins, suggesting HDAC4 has roles beyond transcriptional regulation. To that end, we performed a genetic interaction screen in Drosophila and identified 26 genes that interacted with HDAC4, including Ubc9, the sole SUMO E2-conjugating enzyme. RNA interference-induced reduction of Ubc9 in the adult brain impaired long-term memory in the courtship suppression assay, a Drosophila model of associative memory. We also demonstrate that HDAC4 and Ubc9 interact genetically during memory formation, opening new avenues for investigating the mechanisms through which HDAC4 regulates memory formation and other neurological processes.

A Comparative Study of Sparse Associative Memories

We study various models of associative memories with sparse information, i.e. a pattern to be stored is a random string of $0$s and $1$s with about $\log N$ $1$s, only. We compare different synaptic weights, architectures and retrieval mechanisms to shed light on the influence of the various parameters on the storage capacity.

Protein Folding and Structure Prediction from the Ground Up: The Atomistic Associative Memory, Water Mediated, Structure and Energy Model.

The associative memory, water mediated, structure and energy model (AWSEM) is a coarse-grained force field with transferable tertiary interactions that incorporates local in sequence energetic biases using bioinformatically derived structural information about peptide fragments with locally similar sequences that we call memories. The memory information from the protein data bank (PDB) database guides proper protein folding. The structural information about available sequences in the database varies in quality and can sometimes lead to frustrated free energy landscapes locally. One way out of this difficulty is to construct the input fragment memory information from all-atom simulations of portions of the complete polypeptide chain. In this paper, we investigate this approach first put forward by Kwac and Wolynes in a more complete way by studying the structure prediction capabilities of this approach for six α-helical proteins. This scheme which we call the atomistic associative memory, water mediated, structure and energy model (AAWSEM) amounts to an ab initio protein structure prediction method that starts from the ground up without using bioinformatic input. The free energy profiles from AAWSEM show that atomistic fragment memories are sufficient to guide the correct folding when tertiary forces are included. AAWSEM combines the efficiency of coarse-grained simulations on the full protein level with the local structural accuracy achievable from all-atom simulations of only parts of a large protein. The results suggest that a hybrid use of atomistic fragment memory and database memory in structural predictions may well be optimal for many practical applications.

Holographic Embeddings of Knowledge Graphs

Learning embeddings of entities and relations is an efficient and versatile method to perform machine learning on relational data such as knowledge graphs. In this work, we propose holographic embeddings (HolE) to learn compositional vector space representations of entire knowledge graphs. The proposed method is related to holographic models of associative memory in that it employs circular correlation to create compositional representations. By using correlation as the compositional operator, HolE can capture rich interactions but simultaneously remains efficient to compute, easy to train, and scalable to very large datasets. Experimentally, we show that holographic embeddings are able to outperform state-of-the-art methods for link prediction on knowledge graphs and relational learning benchmark datasets.

Gradient descent learning rule for complex‐valued associative memories with large constant terms

Complex‐valued associative memories (CAMs) are one of the most promising associative memory models by neural networks. However, the low noise tolerance of CAMs is often a serious problem. A projection learning rule with large constant terms improves the noise tolerance of CAMs. However, the projection learning rule can be applied only to CAMs with full connections. In this paper, we propose a gradient descent learning rule with large constant terms, which is not restricted by network topology. We realize large constant terms by regularization to connection weights. By computer simulations, we prove that the proposed learning algorithm improves noise tolerance. © 2016 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

A unified associative memory model based on external inputs of continuous recurrent neural networks

A unified associative memory model with a novel method for designing associative memories is presented in this paper. Based on continuous recurrent neural networks, bipolar patterns inputted from external can cause the output of neural networks to be memorized patterns. In the method, two conditions relevant to external inputs are derived to ensure the network states converge to a stable interval, and an exponential stable criterion is proposed for the network being a bipolar associative memory with higher recall speed. By introducing a tunable slope activation function and considering time delay, the proposed model is general and can recall the memorized patterns in auto-associative and hetero-associative way, while higher robust and more flexible memory can be obtained through the proposed method. Experimental verification demonstrates the effectiveness and generalization of the proposed method.

Concise quantum associative memories with nonlinear search algorithm

The model of quantum associative memories (QAM) we propose here consists in simplifying and generalizing that of Rigui Zhou \etal \cite{zhou2012} who uses the quantum matrix with binary decision diagram and nonlinear search algorithm in his model. It is worth noting that David Rosenbaum put forth the quantum matrix with binary decision diagram \cite{Rosenbaum2010} and Abrams and Llyod did the nonlinear algorithm. \cite{Abrams1998} Our model gives the possibility to retrieve one of the sought states in multi-values retrieving scheme when a measure on the first register is done. It is better than Grover's algorithm and its modified form which need $\mathcal{O}(\sqrt{\frac{2^n} {m}})$ steps when they are used as the retrieval algorithm. $n$ is the number of qubit of the first register and $m$ the number of values $x$ for which $f(x)=1$. As the nonlinearity makes the system highly susceptible to noise, an analysis of the influence of the single qubit noise channels on the Nonlinear Search Algorithm of our model of QAM, shows a fidelity of about $0.7$ whatever the number of qubits present in the first register.

High-Capacity Quantum Associative Memories

We review our models of quantum associative memories that represent the "quantization" of fully coupled neural networks like the Hopfield model. The idea is to replace the classical irreversible attractor dynamics driven by an Ising model with pattern-dependent weights by the reversible rotation of an input quantum state onto an output quantum state consisting of a linear superposition with probability amplitudes peaked on the stored pattern closest to the input in Hamming distance, resulting in a high probability of measuring a memory pattern very similar to the input. The unitary operator implementing this transformation can be formulated as a sequence of one-qubit and two-qubit elementary quantum gates and is thus the exponential of an ordered quantum Ising model with sequential operations and with pattern-dependent interactions, exactly as in the classical case. Probabilistic quantum memories, that make use of postselection of the measurement result of control qubits, overcome the famed linear storage limitation of their classical counterparts because they permit to completely {\it eliminate crosstalk and spurious memories}. The number of control qubits plays the role of an inverse fictitious temperature, the accuracy of pattern retrieval can be tuned by lowering the fictitious temperature under a critical value for quantum content association while the complexity of the retrieval algorithm remains polynomial for any number of patterns polynomial in the number of qubits. These models solve thus the capacity shortage problem of classical associative memories, providing a {\it polynomial improvement} in capacity. The price to pay is the probabilistic nature of information retrieval.

Cross-validation of matching correlation analysis by resampling matching weights

The strength of association between a pair of data vectors is represented by a nonnegative real number, called matching weight. For dimensionality reduction, we consider a linear transformation of data vectors, and define a matching error as the weighted sum of squared distances between transformed vectors with respect to the matching weights. Given data vectors and matching weights, the optimal linear transformation minimizing the matching error is solved by the spectral graph embedding of Yan et al. (2007). This method is a generalization of the canonical correlation analysis, and will be called as matching correlation analysis (MCA). In this paper, we consider a novel sampling scheme where the observed matching weights are randomly sampled from underlying true matching weights with small probability, whereas the data vectors are treated as constants. We then investigate a cross-validation by resampling the matching weights. Our asymptotic theory shows that the cross-validation, if rescaled properly, computes an unbiased estimate of the matching error with respect to the true matching weights. Existing ideas of cross-validation for resampling data vectors, instead of resampling matching weights, are not applicable here. MCA can be used for data vectors from multiple domains with different dimensions via an embarrassingly simple idea of coding the data vectors. This method will be called as cross-domain matching correlation analysis (CDMCA), and an interesting connection to the classical associative memory model of neural networks is also discussed.

Application of emotion affected associative memory based on mood congruency effects for a humanoid

Emotional factor plays an important role in communication. In the field of psychology, it is known that memory and emotions are closely related to each other. In this paper, we present the significance of emotional factors to associative memory in communication and apply it on human---robot interaction problems. Emotional models for the robot partner are developed, and an interactive robot system with a complex-valued multi-directional associative memory model is proposed. We utilize multi-modal information such as object, gesture, voice, and facial expressions to associate the relationships in associative memory, and generate the emotional information for the robot partner. As a result, the robot partner is able to perform various actions depending on the emotional factors. Results from the interactive experiments indicate possibility of suitable information for communication space being provided from the robot partner.

Listen to Your Customers: Insights into Brand Image Using Online Consumer-Generated Product Reviews

ABSTRACTOnline consumer-generated product reviews are a growing phenomenon and have led to the posting of colossal amounts of data by consumers on the Web. These data include consumers’ thoughts, opinions, and feelings about brands and offer firms the opportunity to “listen in” on consumers to get a better understanding of the topics discussed about their brands. Using the human associative memory model as the theoretical framework, the authors introduce an approach to convert online product reviews into meaningful information about brand images using a novel combination of text mining and network analysis methodologies. Following a network-based understanding of brand image, the authors use online product reviews to extract consumers’ brand associations and their interconnections as well as to depict and characterize the network of brand associations. In an empirical study, the authors test the approach and illustrate its managerial usefulness. The suggested approach allows managers to effectively monitor and detect strengths and weaknesses of brand image. Moreover, the proposed approach is one of the first attempts to measure brand image using consumer-generated content by applying text mining and network analysis.

Predictive energy landscapes for folding membrane protein assemblies.

We study the energy landscapes for membrane protein oligomerization using the Associative memory, Water mediated, Structure and Energy Model with an implicit membrane potential (AWSEM-membrane), a coarse-grained molecular dynamics model previously optimized under the assumption that the energy landscapes for folding α-helical membrane protein monomers are funneled once their native topology within the membrane is established. In this study we show that the AWSEM-membrane force field is able to sample near native binding interfaces of several oligomeric systems. By predicting candidate structures using simulated annealing, we further show that degeneracies in predicting structures of membrane protein monomers are generally resolved in the folding of the higher order assemblies as is the case in the assemblies of both nicotinic acetylcholine receptor and V-type Na(+)-ATPase dimers. The physics of the phenomenon resembles domain swapping, which is consistent with the landscape following the principle of minimal frustration. We revisit also the classic Khorana study of the reconstitution of bacteriorhodopsin from its fragments, which is the close analogue of the early Anfinsen experiment on globular proteins. Here, we show the retinal cofactor likely plays a major role in selecting the final functional assembly.

Neural Population Evidence of Functional Heterogeneity along the CA3 Transverse Axis: Pattern Completion versus Pattern Separation

Classical theories of associative memory model CA3 as a homogeneous attractor network because of its strong recurrent circuitry. However, anatomical gradients suggest a functional diversity along the CA3 transverse axis. We examined the neural population coherence along this axis, when the local and global spatial reference frames were put in conflict with each other. Proximal CA3 (near the dentate gyrus), where the recurrent collaterals are the weakest, showed degraded representations, similar to the pattern separation shown by the dentate gyrus. Distal CA3 (near CA2), where the recurrent collaterals are the strongest, maintained coherent representations in the conflict situation, resembling the classic attractor network system. CA2 also maintained coherent representations. This dissociation between proximal and distal CA3 provides strong evidence that the recurrent collateral system underlies the associative network functions of CA3, with a separate role of proximal CA3 in pattern separation.

Electrostatics, structure prediction, and the energy landscapes for protein folding and binding.

While being long in range and therefore weakly specific, electrostatic interactions are able to modulate the stability and folding landscapes of some proteins. The relevance of electrostatic forces for steering the docking of proteins to each other is widely acknowledged, however, the role of electrostatics in establishing specifically funneled landscapes and their relevance for protein structure prediction are still not clear. By introducing Debye-Hückel potentials that mimic long-range electrostatic forces into the Associative memory, Water mediated, Structure, and Energy Model (AWSEM), a transferable protein model capable of predicting tertiary structures, we assess the effects of electrostatics on the landscapes of thirteen monomeric proteins and four dimers. For the monomers, we find that adding electrostatic interactions does not improve structure prediction. Simulations of ribosomal protein S6 show, however, that folding stability depends monotonically on electrostatic strength. The trend in predicted melting temperatures of the S6 variants agrees with experimental observations. Electrostatic effects can play a range of roles in binding. The binding of the protein complex KIX-pKID is largely assisted by electrostatic interactions, which provide direct charge-charge stabilization of the native state and contribute to the funneling of the binding landscape. In contrast, for several other proteins, including the DNA-binding protein FIS, electrostatics causes frustration in the DNA-binding region, which favors its binding with DNA but not with its protein partner. This study highlights the importance of long-range electrostatics in functional responses to problems where proteins interact with their charged partners, such as DNA, RNA, as well as membranes.

Capacity of an associative memory model on random graph architectures

We analyze the storage capacity of the Hopfield models on classes of random graphs. While such a setup has been analyzed for the case that the underlying random graph model is an Erdös–Renyi graph, other architectures, including those investigated in the recent neuroscience literature, have not been studied yet. We develop a notion of storage capacity that highlights the influence of the graph topology and give results on the storage capacity for not too irregular random graph models. The class of models investigated includes the popular power law graphs for some parameter values.

On the capacity of an associative memory model based on neural cliques

Based on recent work by Gripon and Berrou (2011), we introduce a new model of an associative memory. We give upper and lower bounds on the memory capacity of the model.

Water Mediated Interactions and the Protein Folding Phase Diagram in the Temperature-Pressure Plane.

The temperature-pressure behavior of two proteins, ubiquitin and λ-repressor, is explored using a realistically coarse-grained physicochemical model, the associative memory, water mediated, structure and energy model (AWSEM). The phase diagram across the temperature-pressure plane is obtained by perturbing the water mediated interactions in the Hamiltonian systematically. The phase diagrams calculated with direct simulations along with an extended bridge sampling estimator show the main features found experimentally, including both cold- and pressure-denaturation. The denatured ensembles in different parts of the phase diagram are characterized and found to be structurally distinct. The protein energy landscape is found to be funneled throughout the phase diagram, but modest changes in the entropy and free energy of the water are found to drive both cold and pressure induced denaturation.

Oscillator Array Models for Associative Memory and Pattern Recognition

Brain-inspired arrays of parallel processing oscillators represent an intriguing alternative to traditional computational methods for data analysis and recognition. This alternative is now becoming more concrete thanks to the advent of emerging oscillators fabrication technologies providing high density packaging and low power consumption. One challenging issue related to oscillator arrays is the large number of system parameters and the lack of efficient computational techniques for array simulation and performance verification. This paper provides a realistic phase-domain modeling and simulation methodology of oscillator arrays which is able to account for the relevant device nonidealities. The model is employed to investigate the associative memory performance of arrays composed of resonant LC oscillators.