Associative Memory Model Research Articles

Associative memory and recall model with KID model for human activity recognition

Learning, memorizing and recalling knowledge are the basic functions of cognitive models. These models must prioritize which stimulants to respond to as well as package acquired knowledge in an easy to retrieve manner. The human brain is a cognitive model that derives information from sensor data such as vision, associates different patterns to create knowledge, and uses chunking mechanisms to package the acquired knowledge in manageable entities. The use of chunking mechanisms by the brain aids it to overcome its short-term memory (STM) capacity limitation. Through chunking, each entity held in the STM is a chunk containing more associations (knowledge) in it. By mimicking the human brain, this study proposes an associative memory and recall (AMR) model that stores associative knowledge from sensor data. Using chunking mechanisms, AMR can organize human activity knowledge in the manner that is efficient and effective to store and recall. The knowledge–information–data (KID) model is used for learning associative knowledge while the AMR continuously looks for associations among knowledge units and merges related units using merging mechanisms. The chunking mechanisms used in this study are inspired by the chunking mechanisms of the brain i.e. goal oriented chunking and automatic chunking.

Template-Guided Protein Structure Prediction and Refinement Using Optimized Folding Landscape Force Fields.

When good structural templates can be identified, template-based modeling is the most reliable way to predict the tertiary structure of proteins. In this study, we combine template-based modeling with a realistic coarse-grained force field, AWSEM, that has been optimized using the principles of energy landscape theory. The Associative memory, Water mediated, Structure and Energy Model (AWSEM) is a coarse-grained force field having both transferable tertiary interactions and knowledge-based local-in-sequence interaction terms. We incorporate template information into AWSEM by introducing soft collective biases to the template structures, resulting in a model that we call AWSEM-Template. Structure prediction tests on eight targets, four of which are in the low sequence identity "twilight zone" of homology modeling, show that AWSEM-Template can achieve high-resolution structure prediction. Our results also confirm that using a combination of AWSEM and a template-guided potential leads to more accurate prediction of protein structures than simply using a template-guided potential alone. Free energy profile analyses demonstrate that the soft collective biases to the template effectively increase funneling toward native-like structures while still allowing significant flexibility so as to allow for correction of discrepancies between the target structure and the template. A further stage of refinement using all-atom molecular dynamics augmented with soft collective biases to the structures predicted by AWSEM-Template leads to a further improvement of both backbone and side-chain accuracy by maintaining sufficient flexibility but at the same time discouraging unproductive unfolding events often seen in unrestrained all-atom refinement simulations. The all-atom refinement simulations also reduce patches of frustration of the initial predictions. Some of the backbones found among the structures produced during the initial coarse-grained prediction step already have CE-RMSD values of less than 3 Å with 90% or more of the residues aligned to the experimentally solved structure for all targets. All-atom structures generated during the following all-atom refinement simulations, which started from coarse-grained structures that were chosen without reference to any knowledge about the native structure, have CE-RMSD values of less than 2.5 Å with 90% or more of the residues aligned for 6 out of 8 targets. Clustering low energy structures generated during the initial coarse-grained annealing picks out reliably structures that are within 1 Å of the best sampled structures in 5 out of 8 cases. After the all-atom refinement, structures that are within 1 Å of the best sampled structures can be selected using a simple algorithm based on energetic features alone in 7 out of 8 cases.

AWSEM-IDP: A Coarse-Grained Force Field for Intrinsically Disordered Proteins.

The associative memory, water-mediated, structure and energy model (AWSEM) has been successfully used to study protein folding, binding, and aggregation problems. In this work, we introduce AWSEM-IDP, a new AWSEM branch for simulating intrinsically disordered proteins (IDPs), where the weights of the potentials determining secondary structure formation have been finely tuned, and a novel potential is introduced that helps to precisely control both the average extent of protein chain collapse and the chain's fluctuations in size. AWSEM-IDP can efficiently sample large conformational spaces, while retaining sufficient molecular accuracy to realistically model proteins. We applied this new model to two IDPs, demonstrating that AWSEM-IDP can reasonably well reproduce higher-resolution reference data, thus providing the foundation for a transferable IDP force field. Finally, we used thermodynamic perturbation theory to show that, in general, the conformational ensembles of IDPs are highly sensitive to fine-tuning of force field parameters.

Non-convex Multi-species Hopfield Models

In this work we introduce a multi-species generalization of the Hopfield model for associative memory, where neurons are divided into groups and both inter-groups and intra-groups pair-wise interactions are considered, with different intensities. Thus, this system contains two of the main ingredients of modern Deep neural network architectures: Hebbian interactions to store patterns of information and multiple layers coding different levels of correlations. The model is completely solvable in the low-load regime with a suitable generalization of the Hamilton-Jacobi technique, despite the Hamiltonian can be a non-definite quadratic form of the magnetizations. The family of multi-species Hopfield model includes, as special cases, the 3-layers Restricted Boltzmann Machine (RBM) with Gaussian hidden layer and the Bidirectional Associative Memory (BAM) model.

Inhibition of histone deacetylase 3 via RGFP966 facilitates cortical plasticity underlying unusually accurate auditory associative cue memory for excitatory and inhibitory cue-reward associations

Epigenetic mechanisms are key for regulating long-term memory (LTM) and are known to exert control on memory formation in multiple systems of the adult brain, including the sensory cortex. One epigenetic mechanism is chromatin modification by histone acetylation. Blocking the action of histone de-acetylases (HDACs) that normally negatively regulate LTM by repressing transcription has been shown to enable memory formation. Indeed, HDAC inhibition appears to facilitate memory by altering the dynamics of gene expression events important for memory consolidation. However, less understood are the ways in which molecular-level consolidation processes alter subsequent memory to enhance storage or facilitate retrieval. Here we used a sensory perspective to investigate whether the characteristics of memory formed with HDAC inhibitors are different from naturally-formed memory. One possibility is that HDAC inhibition enables memory to form with greater sensory detail than normal. Because the auditory system undergoes learning-induced remodeling that provides substrates for sound-specific LTM, we aimed to identify behavioral effects of HDAC inhibition on memory for specific sound features using a standard model of auditory associative cue-reward learning, memory, and cortical plasticity. We found that three systemic post-training treatments of an HDAC3-inhibitor (RGPF966, Abcam Inc.) in rats in the early phase of training facilitated auditory discriminative learning, changed auditory cortical tuning, and increased the specificity for acoustic frequency formed in memory of both excitatory (S+) and inhibitory (S−) associations for at least 2 weeks. The findings support that epigenetic mechanisms act on neural and behavioral sensory acuity to increase the precision of associative cue memory, which can be revealed by studying the sensory characteristics of long-term associative memory formation with HDAC inhibitors.

Decoding the effects of a product’s cast shadow in brand advertising

PurposeThis research aims to investigate the impact of incorporating product shadows in brand advertising on consumer brand evaluations.Design/methodology/approachThree studies were designed using experimental approach to demonstrate how the presence of a product’s cast shadow in a brand’s promotional frame implicitly influences brand evaluations differently for experiential vs functional brands.FindingsThe presence of a product’s cast shadow in a visual frame implicitly complements abstract processing of an experiential brand but hurts a functional brand’s concrete gestalt by acting as visual noise, thereby improving an experiential brand’s overall evaluation in comparison to a functional brand.Research limitations/implicationsCurrent findings highlight the importance of using appropriate visual elements (especially subtle elements such as product shadows) to ensure communication consistency between the firm-formulated brand concept and the consumer-perceived brand image.Practical implicationsExperiential (vs functional) brand images are harder to build and maintain. Current findings show that a mere presence of the product’s shadow in an experiential (vs functional) brand’s ad frame reinforces the experiential brand image by acting as a consistent element in the experiential brand’s ad frame that enhances the overall ease of product evaluation. Hence, product shadows should be used as strategic tools by brand managers, rather than a random ad-execution choice.Originality/valueThis research makes an initial attempt to explore the relationship between product shadows and consumer brand perceptions. It provides a deeper understanding of the underlying process (based on associative networks memory model, construal level theory and processing fluency model) that influence specific brand perceptions (experiential vs functional) when a product is showcased with its shadow in a promotional frame.

Chaotic Itinerancy in Random Dynamical System Related to Associative Memory Models

We consider a random dynamical system arising as a model of the behavior of a macrovariable related to a more complicated model of associative memory. This system can be seen as a small (stochastic and deterministic) perturbation of a determinstic system having two weak attractors which are destroyed after the perturbation. We show, with a computer aided proof, that the system has a kind of chaotic itineracy. Typical orbits are globally chaotic, while they spend a relatively long time visiting the attractor’s ruins.

Open quantum generalisation of Hopfield neural networks

We propose a new framework to understand how quantum effects may impact on the dynamics of neural networks. We implement the dynamics of neural networks in terms of Markovian open quantum systems, which allows us to treat thermal and quantum coherent effects on the same footing. In particular, we propose an open quantum generalisation of the Hopfield neural network, the simplest toy model of associative memory. We determine its phase diagram and show that quantum fluctuations give rise to a qualitatively new non-equilibrium phase. This novel phase is characterised by limit cycles corresponding to high-dimensional stationary manifolds that may be regarded as a generalisation of storage patterns to the quantum domain.

Developing an ideation method to be used in cross-functional inter-organizational teams by means of action design research

Using ideation methods in an optimal way has a great potential to increase the number of ideas which a team can contribute during the conceptual phase of product development in industry. Previous studies on ideation methods have been mainly statistical studies in a laboratory setting. In the present study, however, the aim was to develop tailor-made ideation methods in a specific context, among actors on a deregulated railway market, through close interaction between researchers and engineers in a real-world context. Considering previous relevant research on ideation methods, associative memory models and the findings from tests of established ideation methods performed in the same group, a number of preliminary design principles were formulated and implemented in an ideation method that combines individual phases of rotational and gallery viewing with phases of verbal group interaction. The method was tested and refined in a cross-functional inter-organizational group comprising participants from different actors in the railway sector. Besides its provision of qualitative and quantitative test results, the present study has considered the opinions of the participants in detail, which can give important insights into the factors determining whether such a method will be implemented in industry. The participants found the method to be more useful and to generate more ideas that could be used in practice compared to the established ideation methods which they had tried. The learning derived from the specific case was formalized into a number of design principles for ideation methods to be used in cross-functional inter-organizational groups.

Interval-valued fuzzy morphological associative memories: Some theoretical aspects and applications

Fuzzy morphological associative memories (FMAMs) are generalizations of many well-known fuzzy associative memory (FAM) models from the literature and have been employed to implement fuzzy rule-based systems. Inspired by the advent of type-2 fuzzy systems and in particular interval type-2 fuzzy systems, we present some theoretical foundations of interval-valued fuzzy morphological associative memories (IV-FMAMs), whose weight matrices can be constructed using representable interval-valued fuzzy operators, and we introduce a novel IV-FMAM approach towards interval type-2 fuzzy inference systems. The paper also includes some experimental results in non-linear function identification as well as time series prediction. These results are compared with the ones produced by some interval and general type-2 fuzzy models from the recent literature.

Singular value Decomposition applied to Associative Memory of Hopfield Neural Network

Content-addressable memories are useful for storage and data retrieval from arrays of sensors. The Hopfield neural network is a model of associative content-addressable memory with a simple flexible structure. Design of this artificial neural network is capable of memorizing large quantity of data and recalling the same from available information. In this work, we have shown that with corrupted input dataset, the correct set of data can be retrieved from approximated or compressed associative memory matrix. The idea has been explained through an experiment using binary datasets. The proposed methodology will increase the storage capacity of associative memory. In the experiment we have shown that it will minimize the number of arithmetic operations involved in recovery process of the Hopfield model but will also ensure correctness of information retrieval from the synaptic matrix from a corrupted dataset as input.

Molecular Associative Memory with Spatial Auto-logistic Model for Pattern Recall

We propose a molecular associative memory model, by combining auto-logistic specifications, which capture statistical dependencies within the local neighborhood systems of the exposed knowledge, with the bio-inspired DNA-based molecular operations, which store and evolve the memory. Our model, characterized by only the local dependencies of the spatial binary data, allows to capture only a fewer features. Our memory model stores the exposed patterns and recalls the stored patterns through bio-inspired molecular operations. Our molecular memory simulation exemplifies the applications of associative memories in pattern storage and retrieval with high recall accuracy, even with lower order memory traces (pair-wise cliques) and thus exhibits brain-like content-addressing cognitive abilities.

A Memristive Neural Network Model With Associative Memory for Modeling Affections

Memristor is a nonlinear resistor with memory, which has the characteristics of neuron synapses and can be used to design a new generation of memristive neural networks. Based on the Pavlov associative memory, a novel memristive associative memory neural network model is designed by using the charge-controlled nanoscale HP memristor model as the electronic synapse. This model includes neurons, memristor synapses, and weighted input feedback learning rule. Based on the proposed memristive associative memory model, a memristor-based neural network structure for simulating human emotions is further designed. The emotion simulation takes into account the excitation and inhibition between different neurons, making it more bionic. In order to simulate the memristive neural network structure, a relatively simplified emotional simulation circuit is constructed, which effectively reduces the network complexity and circuit power consumption. Finally, PSPICE is used to simulate the circuit. The simulation results not only verify the correctness of the model but also achieve a simple simulation of human emotions, which is helpful for the further development of the artificial neural network in the field of artificial intelligence.

Ising formulation of associative memory models and quantum annealing recall

Associative memory models, in theoretical neuro- and computer sciences, can generally store a sublinear number of memories. We show that using quantum annealing for recall tasks endows associative memory models with exponential storage capacities. Theoretically, we obtain the radius of attractor basins, $R(N)$, and the capacity, $C(N)$, of such a scheme and their tradeoffs. Our calculations establish that for randomly chosen memories the capacity of a model using the Hebbian learning rule with recall via quantum annealing is exponential in the size of the problem, $C(N)=\mathcal{O}(e^{C_1N}),~C_1\geq0$, and succeeds on randomly chosen memory sets with a probability of $(1-e^{-C_2N}),~C_2\geq0$ with $C_1+C_2=(.5-f)^2/(1-f)$, where, $f=R(N)/N,~0\leq f\leq .5$ is the radius of attraction in terms of Hamming distance of an input probe from a stored memory as a fraction of the problem size. We demonstrate the application of this scheme on a programmable quantum annealing device - the Dwave processor.

A Comprehensive Study on the Applications of Artificial Intelligence for the Medical Diagnosis and Prognosis of Asthma

An estimated 300 million people worldwide suffer from asthma, and this number is expected to increase to 400 million by 2025. Approximately 250,000 people die prematurely each year from asthma out of which, almost all deaths are avoidable. Most of these deaths occur because the patients are unaware of their asthmatic morbidity. If detected early, asthmatic mortality rate can be reduced by 78%, provided that the patients carry appropriate medication for the same and/or are in lose vicinity to medical equipment like nebulizers. This study focuses on the development and valuation of algorithms to diagnose asthma through symptom intensive questionary, clinical data and medical reports. Machine Learning Algorithms like Back-propagation model, Context Sensitive Auto-Associative Memory Neural Network Model, C4.5 Algorithm, Bayesian Network and Particle Swarm Optimization have been employed for the diagnosis of asthma and later a comparison is made between their respective prospects. All algorithms received an accuracy of over 80%. However, the use of Auto Associative Memory Model (on a layered Artificial Neural Network) displayed much better results. It reached to an accuracy of over 90% and an inconclusive diagnosis rate of less than 1% when trained with adequate data. In the end, naive mobile based applications were developed on Android and iOS that made use of the self-training auto associative memory model to achieve an accuracy of nearly 94.2%.

The Associative Memory, Water Mediated, Structure and Energy Model (AWSEM)-Amylometer: Predicting Amyloid Propensity and Fibril Topology Using an Optimized Folding Landscape Model.

Amyloids are fibrillar protein aggregates with simple repeated structural motifs in their cores, usually β-strands but sometimes α-helices. Identifying the amyloid-prone regions within protein sequences is important both for understanding the mechanisms of amyloid-associated diseases and for understanding functional amyloids. Based on the crystal structures of seven cross-β amyloidogenic peptides with different topologies and one recently solved cross-α fiber structure, we have developed a computational approach for identifying amyloidogenic segments in protein sequences using the Associative memory, Water mediated, Structure and Energy Model (AWSEM). The AWSEM-Amylometer performs favorably in comparison with other predictors in predicting aggregation-prone sequences in multiple data sets. The method also predicts well the specific topologies (the relative arrangement of β-strands in the core) of the amyloid fibrils. An important advantage of the AWSEM-Amylometer over other existing methods is its direct connection with an efficient, optimized protein folding simulation model, AWSEM. This connection allows one to combine efficient and accurate search of protein sequences for amyloidogenic segments with the detailed study of the thermodynamic and kinetic roles that these segments play in folding and aggregation in the context of the entire protein sequence. We present new simulation results that highlight the free energy landscapes of peptides that can take on multiple fibril topologies. We also demonstrate how the Amylometer methodology can be straightforwardly extended to the study of functional amyloids that have the recently discovered cross-α fibril architecture.

Exploring the sources and mechanisms of cognitive errors in medical diagnosis with associative memory models.

One of the central challenges of third millennium medicine is the abatement of medical errors. Among the most frequent and hardiest causes of misdiagnosis are cognitive errors produced by faulty medical reasoning. These errors have been analyzed from the perspectives of cognitive psychology and empirical medical studies. We introduce a neurocognitive model of medical diagnosis to address this issue. We construct a connectionist model based on the associative nature of human memory to explore the non-analytical, pattern-recognition mode of diagnosis. A context-dependent matrix memory associates signs and symptoms with their corresponding diseases. The weights of these associations depend on the frequencies of occurrence of each disease and on the different combinations of signs and symptoms of each presentation of that disease. The system receives signs and symptoms and by a second input, the degree of diagnostic uncertainty. Its output is a probabilistic map on the set of possible diseases. The model reproduces different kinds of well-known cognitive errors in diagnosis. Errors in the model come from two sources. One, dependent on the knowledge stored in memory, varies with the accumulated experience of the physician and explains age-dependent errors and effects such as epidemiological masking. The other is independent of experience and explains contextual effects such as anchoring. Our results strongly suggest that cognitive biases are inevitable consequences of associative storage and recall. We found that this model provides valuable insight into the mechanisms of cognitive error and we hope it will prove useful in medical education.

Dynamics of BAM neural networks with mixed delays and leakage time‐varying delays in the weighted pseudo–almost periodic on time‐space scales

Bidirectional associative memory models are 2‐layer heteroassociative networks. In this work, we prove the existence and the global exponential stability of the unique weighted pseudo–almost periodic solution of bidirectional associative memory neural networks with mixed time‐varying delays and leakage time‐varying delays on time‐space scales. Some sufficient conditions are given for the existence, the convergence, and the global exponential stability of the weighted pseudo–almost periodic solution by using fixed‐point theorem and differential inequality techniques. The results of this paper complement the previous outcomes. An example is given to show the effectiveness of the derived results via computer simulations.

Estimating Memory Deterioration Rates Following Neurodegeneration and Traumatic Brain Injuries in a Hopfield Network Model.

Neurodegenerative diseases and traumatic brain injuries (TBI) are among the main causes of cognitive dysfunction in humans. At a neuronal network level, they both extensively exhibit focal axonal swellings (FAS), which in turn, compromise the information encoded in spike trains and lead to potentially severe functional deficits. There are currently no satisfactory quantitative predictors of decline in memory-encoding neuronal networks based on the impact and statistics of FAS. Some of the challenges of this translational approach include our inability to access small scale injuries with non-invasive methods, the overall complexity of neuronal pathologies, and our limited knowledge of how networks process biological signals. The purpose of this computational study is three-fold: (i) to extend Hopfield's model for associative memory to account for the effects of FAS, (ii) to calibrate FAS parameters from biophysical observations of their statistical distribution and size, and (iii) to systematically evaluate deterioration rates for different memory-recall tasks as a function of FAS injury. We calculate deterioration rates for a face-recognition task to account for highly correlated memories and also for a discrimination task of random, uncorrelated memories with a size at the capacity limit of the Hopfield network. While it is expected that the performance of any injured network should decrease with injury, our results link, for the first time, the memory recall ability to observed FAS statistics. This allows for plausible estimates of cognitive decline for different stages of brain disorders within neuronal networks, bridging experimental observations following neurodegeneration and TBI with compromised memory recall. The work lends new insights to help close the gap between theory and experiment on how biological signals are processed in damaged, high-dimensional functional networks, and towards positing new diagnostic tools to measure cognitive deficits.

High storage capacity in the Hopfield model with auto-interactions—stability analysis

Recent studies point to the potential storage of a large number of patterns in the celebrated Hopfield associative memory model, well beyond the limits obtained previously. We investigate the properties of new fixed points to discover that they exhibit instabilities for small perturbations and are therefore of limited value as associative memories. Moreover, a large deviations approach also shows that errors introduced to the original patterns induce additional errors and increased corruption with respect to the stored patterns.