Associative Memory Research Articles

THE CONSTRUCTION OF INTELLIGENT RULE-BASED SYSTEM WITH DEEP-LEARNING FUNCTIONALITY FOR OIL AND GAS WELLS DRILLING

The article is devoted to solving the problems of predictive modeling using machine learning methods under conditions of uncertainty in huge data sets created in various sectors of the economy, in particular, the oil and gas industry, due to the spread of IoT and cloud services. The author notes that existing machine learning methods, including widely used deep learning methods, face limitations in achieving high prediction accuracy due to various uncertainties. The article explores fuzzy learning approaches to overcome uncertainties such as imprecision and ambiguity, but emphasizes their limitations of incompleteness and vagueness. It is argued that rule-based expert systems augmented with deep learning offer a more robust framework for dealing with such uncertainties by integrating associative memory, helping to produce more accurate predictive models, especially in data-intensive scenarios such as oil and gas drilling processes. The study focuses on improving predictive modeling under conditions of uncertainty in large data sets. We discuss the limitations of fuzzy learning when processing incomplete data and propose a hybrid approach to the problem. Integration of the associative memory learning method into the logical inference procedures allows to identify accurate data models, thereby improving the quality of forecasting under uncertainty. This integrated approach aims to improve forecasting accuracy by leveraging the strengths of deep learning in pattern recognition and the ability of rule-based systems to cope with uncertainty. The main result of the research work is the development of a new method that integrates a level of deep processing with associative memory into a rule-based system to improve forecasting accuracy under uncertainty. This was achieved by using a multilayer neural network and adding additional parameters. This method significantly improved the accuracy of predicting the dynamics of operational parameters during drilling. Thus, the presented study is a new contribution to the task of developing forecasting methods under conditions of uncertainty, which is especially relevant for oil and gas drilling.

Long-term abacus training gains in children are predicted by medial temporal lobe anatomy and circuitry.

Abacus-based mental calculation (AMC) is a widely used educational tool for enhancing math learning, offering an accessible and cost-effective method for classroom implementation. Despite its universal appeal, the neurocognitive mechanisms that drive the efficacy of AMC training remain poorly understood. Notably, although abacus training relies heavily on the rapid recall of number positions and sequences, the role of memory systems in driving long-term AMC learning remains unknown. Here, we sought to address this gap by investigating the role of the medial temporal lobe (MTL) memory system in predicting long-term AMC training gains in second-grade children, who were longitudinally assessed up to fifth grade. Leveraging multimodal neuroimaging data, we tested the hypothesis that MTL systems, known for their involvement in associative memory, are instrumental in facilitating AMC-induced improvements in math skills. We found that gray matter volume in bilateral MTL, along with functional connectivity between the MTL and frontal and ventral temporal-occipital cortices, significantly predicted learning gains. Intriguingly, greater gray matter volume but weaker connectivity of the posterior parietal cortex predicted better learning outcomes, offering a more nuanced view of brain systems at play in AMC training. Our findings not only underscore the critical role of the MTL memory system in AMC training but also illuminate the neurobiological factors contributing to individual differences in cognitive skill acquisition. RESEARCH HIGHLIGHTS: We investigated the role of medial temporal lobe (MTL) memory system in driving children's math learning following abacus-based mental calculation (AMC) training. AMC training improved math skills in elementary school children across their second and fifth grade. MTL structural integrity and functional connectivity with prefrontal and ventral temporal-occipital cortices predicted long-term AMC training-related gains.

Topological properties of basins of attraction of width bounded autoencoders

In [A. Radhakrishnan, M. Belkin and C. Uhler, Overparameterized neural networks implement associative memory, Proc. Natl. Acad. Sci. USA 117(44) (2020) 27162–27170], the authors empirically show that autoencoders trained with standard SGD methods form basins of attraction around their training data. We consider network functions of width not exceeding the input dimension and prove that in this situation, such basins of attraction are bounded and their complement cannot have bounded components. Our conditions in these results are met in several experiments reported in [A. Radhakrishnan, M. Belkin and C. Uhler, Overparameterized neural networks implement associative memory, Proc. Natl. Acad. Sci. USA 117(44) (2020) 27162–27170] and we thus address a question posed therein. We also show that under some more restrictive conditions, the basins of attraction are path-connected. The necessity of the conditions in our results is demonstrated by means of examples.

Examining the neural basis of unitization: A review.

Associative memory refers to the ability to form and remember associations between individual pieces of information rather than memory for a single object or word. Encoding associations in memory tends to be a more difficult task than item (only) encoding, because associative memory requires encoding multiple items as well as the specific links amongst the items. Accordingly, researchers have worked to identify interventions and strategies to reduce the effort and neural resources required for successful associative memory processing. Unitization is one such strategy that has traditionally been defined as the process by which two or more discrete items are processed, or encoded, such that they are perceived as a single ensemble. The current review explores the neural research on unitization while considering the behavioral benefits that accompany the process.

ANALYSIS OF CHANGES IN SENSORY PREFERENCES OF TOURISTS UNDER MARTIAL LAW

The study of changes in sensory preferences of tourists under martial law is important both for understanding the impact of military conflicts on people's psychology and behaviour, and for developing strategies for managing the tourism industry in conditions of instability and crisis. The aim of the study is to analyse the changes in tourists' sensory preferences due to the effects of the Russian-Ukrainian war. Two sociological surveys were used to analyse and draw conclusions on the changes in the sensory preferences of tourists in connection with the large-scale invasion of Russia. The first survey was conducted in November-December 2021 and January 2022, the second in September and October 2023, which was a continuation of the first. A chi-squared test was calculated to confirm the hypothesis of a difference in the answers given by respondents before and during the war. Confirmation of the hypothesis allowed to prove that when developing tourism products, companies should take into account changes in the sensory preferences of tourists. The study showed significant changes in the choice of tourist options by travellers under the influence of sensory sensations that were formed during the Russian-Ukrainian war. The cost of travel has become more important, cultural and walking tourism has lost popularity, while extreme and gastronomic tourism has gained popularity. Aromas and tastes have a greater impact on travellers' associative memory, while visual impressions remain important, with historical themes becoming more popular. It was concluded that the sensory preferences and emotional reactions of travellers differ significantly by age. Younger respondents (under 16 years old) prefer food and music the most, while the older group (over 50 years old) prefers cuisine, tastes and appearance. It is clear that tourism businesses need to adjust their services and marketing strategies to take into account these age-specific characteristics and differences in tourists' reactions to sensory stimuli. Tourism businesses can use the findings of this study to adapt their marketing strategies and tourist offerings to the changing sensory preferences and needs of tourists caused by the military conflict. This will allow them to attract and retain customers even in times of insecurity, instability and crisis.

The anterior retrosplenial cortex is required for short-term object in place recognition memory retrieval: Role of ionotropic glutamate receptors in male and female Long-Evans rats.

The anterior retrosplenial cortex (aRSC) integrates multimodal sensory information into cohesive associative recognition memories. Little is known about how information is integrated during different learning phases (i.e., encoding and retrieval). Additionally, sex differences are observed in performance of some visuospatial memory tasks; however, inconsistent findings warrant more research. We conducted three experiments using the 1-h delay object-in-place (1-h OiP) test to assess recognition memory retrieval in male and female Long-Evans rats. (i) We found both sexes performed equally in three repeated 1-h OiP test sessions. (ii) We showed infusions of a mixture of muscimol/baclofen (GABAA/B receptor agonists) into the aRSC ~15-min prior to the test phase disrupted 1-h OiP in both sexes. (iii) We assessed the role of aRSC ionotropic glutamate receptors in 1-h OiP retrieval using another squad of cannulated rats and confirmed that infusions of either the competitive AMPA/Kainate receptor antagonist CNQX (3 mM) or competitive NMDA receptor antagonist AP-5 (30 mM) (volumes = 0.50 uL/side) significantly impaired 1-h OiP retrieval in both sexes compared to controls. Taken together, findings challenge reported sex differences and clearly establish a role for aRSC ionotropic glutamate receptors in short-term visuospatial recognition memory retrieval. Thus, modulating neural activity in the aRSC may alleviate some memory processing impairments in related disorders.

The ventral hippocampus is activated in olfactory but not auditory threat memory.

Hippocampal networks required for associative memory formation are involved in cue- and context-dependent threat conditioning. The hippocampus is functionally heterogeneous at its dorsal and ventral poles, and recent investigations have focused on the specific roles required from each sub-region for associative conditioning. Cumulative evidence suggests that contextual and emotional information is processed by the dorsal and ventral hippocampus, respectively. However, it is not well understood how these two divisions engage in threat conditioning with cues of different sensory modalities. Here, we compare the involvement of the dorsal and ventral hippocampus in two types of threat conditioning: olfactory and auditory. Our results suggest that the dorsal hippocampus encodes contextual information and is activated upon recall of an olfactory threat memory only if contextual cues are relevant to the threat. Overnight habituation to the context eliminates dorsal hippocampal activation, implying that this area does not directly support cue-dependent threat conditioning. The ventral hippocampus is activated upon recall of olfactory, but not auditory, threat memory regardless of habituation duration. Concurrent activation of the piriform cortex is consistent with its direct connection with the ventral hippocampus. Together, our study suggests a unique role of the ventral hippocampus in olfactory threat conditioning.

The reach of reactivation: Effects of consciously triggered versus unconsciously triggered reactivation of associative memory

Consolidating memories for long-term storage depends on reactivation. Reactivation occurs both consciously, during wakefulness, and unconsciously, during wakefulness and sleep. While considerable work has examined conscious awake and unconscious sleep reactivation, in this study, we directly compare the consequences of conscious and unconscious reactivation during wakefulness. Forty-one participants learned associations consisting of adjective-object-position triads. Objects were clustered into distinct semantic groups (e.g., fruits, vehicles) such that we could examine consequences of reactivation on semantically related memories. After an intensive learning protocol, we systematically reactivated some of the triads by presenting the adjective as a cue. Reactivation was done so that it was consciously experienced for some triads, and only unconsciously processed for others. Memory for spatial positions, the most distal part of the association, was affected by reactivation in a consciousness-dependent and memory-strength-dependent manner. Conscious reactivation resulted in weakening of semantically related memories that were strong initially, resonating with prior findings of retrieval-induced forgetting. Unconscious reactivation, on the other hand, selectively benefited weak reactivated memories, as previously shown for reactivation during sleep. Semantically linked memories were not impaired, but rather were integrated with the reactivated memory. These results taken together demonstrate that conscious and unconscious reactivation have qualitatively different consequences. Results support a consciousness-dependent inhibition account, whereby unconscious reactivation entails less inhibition than conscious reactivation, thus allowing more liberal spread of activation. Findings set the stage for additional exploration into the role of conscious experience in memory storage and structuring.

Global exponential stability of quaternion bidirectional associative memory neural networks with multiple delays

In this paper, the global exponential stability of quaternion BAM neural networks with mixed delays is investigated. The involved delays include multiple time‐varying transmission delays and unbounded distributed delays. Existence and uniqueness of equilibrium point of the considered neural networks are investigated by using the homeomorphism theory between topology spaces. To show sufficient conditions for global exponential stability of the unique equilibrium point, a method based on estimation of solutions is presented. The acquired stability conditions contain only several simple linear scalar inequalities, which can be easily solved. The availability of theoretical results is shown by a numerical example. A point worth emphasizing is that the method based on estimation of solutions is applicable for models of many time‐delay systems after a minor modification.

Ripple-locked coactivity of stimulus-specific neurons and human associative memory

Associative memory enables the encoding and retrieval of relations between different stimuli. To better understand its neural basis, we investigated whether associative memory involves temporally correlated spiking of medial temporal lobe (MTL) neurons that exhibit stimulus-specific tuning. Using single-neuron recordings from patients with epilepsy performing an associative object–location memory task, we identified the object-specific and place-specific neurons that represented the separate elements of each memory. When patients encoded and retrieved particular memories, the relevant object-specific and place-specific neurons activated together during hippocampal ripples. This ripple-locked coactivity of stimulus-specific neurons emerged over time as the patients’ associative learning progressed. Between encoding and retrieval, the ripple-locked timing of coactivity shifted, suggesting flexibility in the interaction between MTL neurons and hippocampal ripples according to behavioral demands. Our results are consistent with a cellular account of associative memory, in which hippocampal ripples coordinate the activity of specialized cellular populations to facilitate links between stimuli.

Examining the relationship between brain activation and proxies of disease severity using quantile regression in individuals at risk of Alzheimer's disease

Previous studies have reported a pattern of hyperactivation in the pre-dementia phase of Alzheimer's disease (AD), followed by hypoactivation in later stages of the disease. This pattern was modeled as an inverse U-shape function between activation and markers of disease severity. In this study, we used quantile regression to model the association between task-related brain activation in AD signature regions and three markers of disease severity (hippocampal volume, cortical thickness, and associative memory). This approach offers distinct advantages over standard regression models as it analyzes the relationship between brain activation and disease severity across various levels of brain activation. Participants were 54 older adults with subjective cognitive decline+ (SCD+) or mild cognitive impairment (MCI) from the CIMA-Q cohort. The analysis revealed an inverse U-shape quadratic function depicting the relationship between disease severity markers and the activation of the left superior parietal region, while a linear relationship was observed for activation of the hippocampal and temporal regions. Quantile differences were observed for temporal and parietal activation, with more pronounced effects observed in the higher quantiles of activation. When comparing quantiles, we found that higher quantile of activation featured a greater number of individuals with SCD+ compared to mild cognitive impairment (MCI). Results are globally consistent with the presence of an inverse-U shape function of activation in relation to disease severity. They study also underscores the utility of employing quantile regression modeling as the modeling approach revealed the presence of non-homogeneous effects across various quantiles.

Exponential Capacity of Dense Associative Memories.

Recent generalizations of the Hopfield model of associative memories are able to store a number P of random patterns that grows exponentially with the number N of neurons, P=exp(αN). Besides the huge storage capacity, another interesting feature of these networks is their connection to the attention mechanism which is part of the Transformer architecture widely applied in deep learning. In this work, we study a generic family of pattern ensembles using a statistical mechanics analysis which gives exact asymptotic thresholds for the retrieval of a typical pattern, α_{1}, and lower bounds for the maximum of the load α for which all patterns can be retrieved, α_{c}, as well as sizes of attraction basins. We discuss in detail the cases of Gaussian and spherical patterns, and show that they display rich and qualitatively different phase diagrams.

The decimation scheme for symmetric matrix factorization

Matrix factorization is an inference problem that has acquired importance due to its vast range of applications that go from dictionary learning to recommendation systems and machine learning with deep networks. The study of its fundamental statistical limits represents a true challenge, and despite a decade-long history of efforts in the community, there is still no closed formula able to describe its optimal performances in the case where the rank of the matrix scales linearly with its size. In the present paper, we study this extensive rank problem, extending the alternative ‘decimation’ procedure that was recently introduced by Camilli and Mézard, and carry out a thorough study of its performance. Decimation aims at recovering one column/line of the factors at a time, by mapping the problem into a sequence of neural network models of associative memory at a tunable temperature. The main advantage of decimation is that its theoretical performance can be studied using statistical physics techniques. In this paper we provide the general analysis applying to a large class of compactly supported priors and we show that the replica symmetric free entropy of the neural network models takes a universal form in the low temperature limit. We then study the decimation phase diagrams in two concrete cases, the sparse Ising prior and the uniform prior, for which we show that matrix factorization is theoretically possible when the ration P/N of rank to variable is below a certain threshold. This suggest that a possible route to solving algorithmically the matrix factorization problem could be to find efficient decimation algorithms; in this respect, we show that simple simulated annealing, though being effective for limited signal sizes, is not scalable.

Hardware acceleration of DNA pattern matching using analog resistive CAMs

DNA pattern matching is essential for many widely used bioinformatics applications. Disease diagnosis is one of these applications since analyzing changes in DNA sequences can increase our understanding of possible genetic diseases. The remarkable growth in the size of DNA datasets has resulted in challenges in discovering DNA patterns efficiently in terms of run time and power consumption. In this paper, we propose an efficient pipelined hardware accelerator that determines the chance of the occurrence of repeat-expansion diseases using DNA pattern matching. The proposed design parallelizes the DNA pattern matching task using associative memory realized with analog content-addressable memory and implements an algorithm that returns the maximum number of consecutive occurrences of a specific pattern within a DNA sequence. We fully implement all the required hardware circuits with PTM 45-nm technology, and we evaluate the proposed architecture on a practical human DNA dataset. The results show that our design is energy-efficient and accelerates the DNA pattern matching task by more than 100× compared to the approaches described in the literature.

Hebbian dreaming for small datasets

The dreaming Hopfield model constitutes a generalization of the Hebbian paradigm for neural networks, that is able to perform on-line learning when “awake” and also to account for off-line “sleeping” mechanisms. The latter have been shown to enhance storing in such a way that, in the long sleep-time limit, this model can reach the maximal storage capacity achievable by networks equipped with symmetric pairwise interactions. In this paper, we inspect the minimal amount of information that must be supplied to such a network to guarantee a successful generalization, and we test it both on random synthetic and on standard structured datasets (i.e., MNIST, Fashion-MNIST and Olivetti). By comparing these minimal thresholds of information with those required by the standard (i.e., always “awake”) Hopfield model, we prove that the present network can save up to ∼90% of the dataset size, yet preserving the same performance of the standard counterpart. This suggests that sleep may play a pivotal role in explaining the gap between the large volumes of data required to train artificial neural networks and the relatively small volumes needed by their biological counterparts. Further, we prove that the model Cost function (typically used in statistical mechanics) admits a representation in terms of a standard Loss function (typically used in machine learning) and this allows us to analyze its emergent computational skills both theoretically and computationally: a quantitative picture of its capabilities as a function of its control parameters is achieved and consistency between the two approaches is highlighted.The resulting network is an associative memory for pattern recognition tasks that learns from examples on-line, generalizes correctly (in suitable regions of its control parameters) and optimizes its storage capacity by off-line sleeping: such a reduction of the training cost can be inspiring toward sustainable AI and in situations where data are relatively sparse.

Hippocampome.org 2.0 is a knowledge base enabling data-driven spiking neural network simulations of rodent hippocampal circuits.

Hippocampome.org is a mature open-access knowledge base of the rodent hippocampal formation focusing on neuron types and their properties. Previously, Hippocampome.org v1.0 established a foundational classification system identifying 122 hippocampal neuron types based on their axonal and dendritic morphologies, main neurotransmitter, membrane biophysics, and molecular expression (Wheeler et al., 2015). Releases v1.1 through v1.12 furthered the aggregation of literature-mined data, including among others neuron counts, spiking patterns, synaptic physiology, in vivo firing phases, and connection probabilities. Those additional properties increased the online information content of this public resource over 100-fold, enabling numerous independent discoveries by the scientific community. Hippocampome.org v2.0, introduced here, besides incorporating over 50 new neuron types, now recenters its focus on extending the functionality to build real-scale, biologically detailed, data-driven computational simulations. In all cases, the freely downloadable model parameters are directly linked to the specific peer-reviewed empirical evidence from which they were derived. Possible research applications include quantitative, multiscale analyses of circuit connectivity and spiking neural network simulations of activity dynamics. These advances can help generate precise, experimentally testable hypotheses and shed light on the neural mechanisms underlying associative memory and spatial navigation.

Hippocampome.org 2.0 is a knowledge base enabling data-driven spiking neural network simulations of rodent hippocampal circuits

Hippocampome.org is a mature open-access knowledge base of the rodent hippocampal formation focusing on neuron types and their properties. Previously, Hippocampome.org v1.0 established a foundational classification system identifying 122 hippocampal neuron types based on their axonal and dendritic morphologies, main neurotransmitter, membrane biophysics, and molecular expression (Wheeler et al., 2015). Releases v1.1 through v1.12 furthered the aggregation of literature-mined data, including among others neuron counts, spiking patterns, synaptic physiology, in vivo firing phases, and connection probabilities. Those additional properties increased the online information content of this public resource over 100-fold, enabling numerous independent discoveries by the scientific community. Hippocampome.org v2.0, introduced here, besides incorporating over 50 new neuron types, now recenters its focus on extending the functionality to build real-scale, biologically detailed, data-driven computational simulations. In all cases, the freely downloadable model parameters are directly linked to the specific peer-reviewed empirical evidence from which they were derived. Possible research applications include quantitative, multiscale analyses of circuit connectivity and spiking neural network simulations of activity dynamics. These advances can help generate precise, experimentally testable hypotheses and shed light on the neural mechanisms underlying associative memory and spatial navigation.

Cultural Perspectives on Basketball Artificial Intelligence Assistant Referee Mode: A Research Approach with Associative Memory Neural Network

Cultural Perspectives on Basketball Artificial Intelligence Assistant Referee Mode: A Research Approach with Associative Memory Neural Network

Dynamic predictive coding: A model of hierarchical sequence learning and prediction in the neocortex.

We introduce dynamic predictive coding, a hierarchical model of spatiotemporal prediction and sequence learning in the neocortex. The model assumes that higher cortical levels modulate the temporal dynamics of lower levels, correcting their predictions of dynamics using prediction errors. As a result, lower levels form representations that encode sequences at shorter timescales (e.g., a single step) while higher levels form representations that encode sequences at longer timescales (e.g., an entire sequence). We tested this model using a two-level neural network, where the top-down modulation creates low-dimensional combinations of a set of learned temporal dynamics to explain input sequences. When trained on natural videos, the lower-level model neurons developed space-time receptive fields similar to those of simple cells in the primary visual cortex while the higher-level responses spanned longer timescales, mimicking temporal response hierarchies in the cortex. Additionally, the network's hierarchical sequence representation exhibited both predictive and postdictive effects resembling those observed in visual motion processing in humans (e.g., in the flash-lag illusion). When coupled with an associative memory emulating the role of the hippocampus, the model allowed episodic memories to be stored and retrieved, supporting cue-triggered recall of an input sequence similar to activity recall in the visual cortex. When extended to three hierarchical levels, the model learned progressively more abstract temporal representations along the hierarchy. Taken together, our results suggest that cortical processing and learning of sequences can be interpreted as dynamic predictive coding based on a hierarchical spatiotemporal generative model of the visual world.

Active oscillatory associative memory

Traditionally, physical models of associative memory assume conditions of equilibrium. Here, we consider a prototypical oscillator model of associative memory and study how active noise sources that drive the system out of equilibrium, as well as nonlinearities in the interactions between the oscillators, affect the associative memory properties of the system. Our simulations show that pattern retrieval under active noise is more robust to the number of learned patterns and noise intensity than under passive noise. To understand this phenomenon, we analytically derive an effective energy correction due to the temporal correlations of active noise in the limit of short correlation decay time. We find that active noise deepens the energy wells corresponding to the patterns by strengthening the oscillator couplings, where the more nonlinear interactions are preferentially enhanced. Using replica theory, we demonstrate qualitative agreement between this effective picture and the retrieval simulations. Our work suggests that the nonlinearity in the oscillator couplings can improve memory under nonequilibrium conditions.