Hierarchical Associative Memory Research Articles

Neural Network Modeling and Organization of a Hierarchical Associative Memory System

Neural Network Modeling and Organization of a Hierarchical Associative Memory System

Hierarchical Associative Memory Model for Artificial General-Purpose Cognitive Agents

This paper presents a model of hierarchical associative memory, which can be used as a basis for building artificial cognitive agents of general purpose. With the help of this model, one of the most important problems of modern machine learning and artificial intelligence in general can be solved — the ability for a cognitive agent to use "life experience" to process the context of the situation in which he was, is and, possibly, will be. This model is applicable for artificial cognitive agents functioning both in specially designed virtual worlds and in objective reality. The use of hierarchical associative memory as a long-term memory of artificial cognitive agents will allow the latter to effectively navigate both in the general knowledge accumulated by mankind and in their life experience. The novelty of the presented work is based on the author’s approach to the construction of context-dependent artificial cognitive agents using an interdisciplinary approach, in particular, based on the achievements of artificial intelligence, cognitology, neurophysiology, psychology and sociology. The relevance of this work is based on the keen interest of the scientific community and the high social demand for the creation of general-level artificial intelligence systems. Associative hierarchical memory, based on the use of an approach similar to the hypercolumns of the human cerebral cortex, is becoming one of the important components of an artificial intelligent agent of the general level. The article will be of interest to all researchers working in the field of building artificial cognitive agents and related fields.

EBrainII: a 3\xa0kW Realtime Custom 3D DRAM Integrated ASIC Implementation of a Biologically Plausible Model of a Human Scale Cortex

The Artificial Neural Networks (ANNs), like CNN/DNN and LSTM, are not biologically plausible. Despite their initial success, they cannot attain the cognitive capabilities enabled by the dynamic hierarchical associative memory systems of biological brains. The biologically plausible spiking brain models, e.g., cortex, basal ganglia, and amygdala, have a greater potential to achieve biological brain like cognitive capabilities. Bayesian Confidence Propagation Neural Network (BCPNN) is a biologically plausible spiking model of the cortex. A human-scale model of BCPNN in real-time requires 162 TFlop/s, 50 TBs of synaptic weight storage to be accessed with a bandwidth of 200 TBs. The spiking bandwidth is relatively modest at 250 GBs/s. A hand-optimized implementation of rodent scale BCPNN has been done on Tesla K80 GPUs require 3 kWs, we extrapolate from that a human scale network will require 3 MWs. These power numbers rule out such implementations for field deployment as cognition engines in embedded systems.The key innovation that this paper reports is that it is feasible and affordable to implement real-time BCPNN as a custom tiled application-specific integrated circuit (ASIC) in 28 nm technology with custom 3D DRAM - eBrainII - that consumes 3 kW for human scale and 12 watts for rodent scale. Such implementations eminently fulfill the demands for field deployment.

Non-Boolean Associative Processing: Circuits, System Architecture, and Algorithms

We present the design and the performance of a hierarchical associative memory (AM) based on phase locking of coupled oscillators used for pattern recognition. The use of coupled oscillators, rather than Boolean logic, provides for implementations using emerging nanotechnology, such as magnetic spin-torque oscillators and resonant body transistor oscillators, which have the potential of lower energy and higher density than CMOS solutions. We develop a model for the general behavior of weakly coupled nonlinear oscillators that perform pattern matching using a simulation of coupled CMOS ring oscillators. We derive a simple analytic model for their phase locking behavior and use this reduced model in a hierarchical AM for image recognition tasks, such as identifying handwritten numbers.

Fully Binary Neural Network Model and Optimized Hardware Architectures for Associative Memories

Brain processes information through a complex hierarchical associative memory organization that is distributed across a complex neural network. The GBNN associative memory model has recently been proposed as a new class of recurrent clustered neural network that presents higher efficiency than the classical models. In this article, we propose computational simplifications and architectural optimizations of the original GBNN. This work leads to significant complexity and area reduction without affecting neither memorizing nor retrieving performance. The obtained results open new perspectives in the design of neuromorphic hardware to support large-scale general-purpose neural algorithms.

Tree-like hierarchical associative memory structures

In this letter we explore an alternative structural representation for Steinbuch-type binary associative memories. These networks offer very generous storage capacities (both asymptotic and finite) at the expense of sparse coding. However, the original retrieval prescription performs a complete search on a fully-connected network, whereas only a small fraction of units will eventually contain desired results due to the sparse coding requirement. Instead of modelling the network as a single layer of neurons we suggest a hierarchical organization where the information content of each memory is a successive approximation of one another. With such a structure it is possible to enhance retrieval performance using a progressively deepening procedure. To backup our intuition we provide collected experimental evidence alongside comments on eventual biological plausibility.

Erratum to: Dynamics control of semantic processes in a hierarchical associative memory

A neural mechanism for control of dynamics and function of associative processes in a hierarchical memory system is demonstrated. For the representation and processing of abstract knowledge, the semantic declarative memory system of the human brain is considered. The dynamics control mechanism is based on the influence of neuronal adaptation on the complexity of neural network dynamics. Different dynamical modes correspond to different levels of the ultrametric structure of the hierarchical memory being invoked during an associative process. The mechanism is deterministic but may also underlie free associative thought processes. The formulation of an abstract neural network model of hierarchical associative memory utilizes a recent approach to incorporate neuronal adaptation. It includes a generalized neuronal activation function recently derived by a Hodgkin-Huxley-type model. It is shown that the extent to which a hierarchically organized memory structure is searched is controlled by the neuronal adaptability, i.e. the strength of coupling between neuronal activity and excitability. In the brain, the concentration of various neuromodulators in turn can regulate the adaptability. An autonomously controlled sequence of bifurcations, from an initial exploratory to a final retrieval phase, of an associative process is shown to result from an activity-dependent release of neuromodulators. The dynamics control mechanism may be important in the context of various disorders of the brain and may also extend the range of applications of artificial neural networks.

Dynamics control of semantic processes in a hierarchical associative memory

A neural mechanism for control of dynamics and function of associative processes in a hierarchical memory system is demonstrated. For the representation and processing of abstract knowledge, the semantic declarative memory system of the human brain is considered. The dynamics control mechanism is based on the influence of neuronal adaptation on the complexity of neural network dynamics. Different dynamical modes correspond to different levels of the ultrametric structure of the hierarchical memory being invoked during an associative process. The mechanism is deterministic but may also underlie free associative thought processes. The formulation of an abstract neural network model of hierarchical associative memory utilizes a recent approach to incorporate neuronal adaptation. It includes a generalized neuronal activation function recently derived by a Hodgkin-Huxley-type model. It is shown that the extent to which a hierarchically organized memory structure is searched is controlled by the neuronal adaptability, i.e. the strength of coupling between neuronal activity and excitability. In the brain, the concentration of various neuromodulators in turn can regulate the adaptability. An autonomously controlled sequence of bifurcations, from an initial exploratory to a final retrieval phase, of an associative process is shown to result from an activity-dependent release of neuromodulators. The dynamics control mechanism may be important in the context of various disorders of the brain and may also extend the range of applications of artificial neural networks.

Dynamics control of semantic processes in a hierarchical associative memory

Dynamics control of semantic processes in a hierarchical associative memory

Self-organization of day cycle and hierarchical associative memory in "live" neural network.

The "live" neural network model is proposed on the basis of "live" neuron model and optimal learning rule. By means of numerical simulation the initial stages of neural network self-organization have been shown: (1) the formation of two activity forms, which are identified with sleep and awaking, and (2) the self-organization of hierarchical associative memory when feeding a receptor excitation to the neural network. The energetic profit of self-organization is demonstrated. The formation of neural ensembles, playing the role of generalized neurons, is obtained.

Naming pictures

Pictures are inherently ambiguous but people categorize and name pictured objects with remarkable consistency. However, the time to assign a name to a picture depends on a large number of variables, ranging from the quality of the picture itself to the level of hierarchy and frequency of the name. We review the empirical results in the psychological literature on how people name pictured objects, summarizing the major variables that affect the name assigned and the time spent assigning it. The underlying regularities in these data are explained by properties of three mechanisms used in picture naming: bottom-up perceptual encoding; hierarchical associative memory; and top-down knowledge-based search. The properties ascribed to these mechanisms are hypothesized on the basis of computational analyses and considerations of characteristics of the neural systems underlying vision.

Storage reallocation in hierarchical associative memories

article Free AccessStorage reallocation in hierarchical associative memories Share on Author: Jeffrey L. Gertz Bell Telephone Laboratories, Incorporated, Holmdel, New Jersey Bell Telephone Laboratories, Incorporated, Holmdel, New JerseyView Profile Authors Info & Claims ACM SIGOPS Operating Systems ReviewVolume 6Issue 1/2June 1972 pp 58–63https://doi.org/10.1145/850614.850625Online:01 June 1972Publication History 0citation43DownloadsMetricsTotal Citations0Total Downloads43Last 12 Months0Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF