Biological Memories Research Articles

Classification and generation of real-world data with an associative memory model

Drawing from memory the face of a friend you have not seen in years is a difficult task. However, if you happen to cross paths, you would easily recognize each other. The biological memory is equipped with an impressive compression algorithm that can store the essential, and then infer the details to match perception. The Willshaw Memory is a simple abstract model for cortical computations which implements mechanisms of biological memories. Using our recently proposed sparse coding prescription for visual patterns [34], this model can store and retrieve an impressive amount of real-world data in a fault-tolerant manner. In this paper, we extend the capabilities of the basic Associative Memory Model by using a Multiple-Modality framework. In this setting, the memory stores several modalities (e.g., visual, or textual) of each pattern simultaneously. After training, the memory can be used to infer missing modalities when just a subset is perceived. Using a simple encoder-memory-decoder architecture, and a newly proposed iterative retrieval algorithm for the Willshaw Model, we perform experiments on the MNIST dataset. By storing both the images and labels as modalities, a single Memory can be used not only to retrieve and complete patterns but also to classify and generate new ones. We further discuss how this model could be used for other learning tasks, thus serving as a biologically-inspired framework for learning.

Manifold epigenetics: A conceptual model that guides engineering strategies to improve whole-body regenerative health.

Despite the promising advances in regenerative medicine, there is a critical need for improved therapies. For example, delaying aging and improving healthspan is an imminent societal challenge. Our ability to identify biological cues as well as communications between cells and organs are keys to enhance regenerative health and improve patient care. Epigenetics represents one of the major biological mechanisms involving in tissue regeneration, and therefore can be viewed as a systemic (body-wide) control. However, how epigenetic regulations concertedly lead to the development of biological memories at the whole-body level remains unclear. Here, we review the evolving definitions of epigenetics and identify missing links. We then propose our Manifold Epigenetic Model (MEMo) as a conceptual framework to explain how epigenetic memory arises and discuss what strategies can be applied to manipulate the body-wide memory. In summary we provide a conceptual roadmap for the development of new engineering approaches to improve regenerative health.

Palimpsest memories stored in memristive synapses.

Biological synapses store multiple memories on top of each other in a palimpsest fashion and at different time scales. Palimpsest consolidation is facilitated by the interaction of hidden biochemical processes governing synaptic efficacy during varying lifetimes. This arrangement allows idle memories to be temporarily overwritten without being forgotten, while previously unseen memories are used in the short term. While embedded artificial intelligence can greatly benefit from this functionality, a practical demonstration in hardware is missing. Here, we show how the intrinsic properties of metal-oxide volatile memristors emulate the processes supporting biological palimpsest consolidation. Our memristive synapses exhibit an expanded doubled capacity and protect a consolidated memory while up to hundreds of uncorrelated short-term memories temporarily overwrite it, without requiring specialized instructions. We further demonstrate this technology in the context of visual working memory. This showcases how emerging memory technologies can efficiently expand the capabilities of artificial intelligence hardware toward more generalized learning memories.

Sparse Associative Memory.

It is still unknown how associative biological memories operate. Hopfield networks are popular models of associative memory, but they suffer from spurious memories and low efficiency. Here, we present a new model of an associative memory that overcomes these deficiencies. We call this model sparse associative memory (SAM) because it is based on sparse projections from neural patterns to pattern-specific neurons. These sparse projections have been shown to be sufficient to uniquely encode a neural pattern. Based on this principle, we investigate theoretically and in simulation our SAM model, which turns out to have high memory efficiency and a vanishingly small probability of spurious memories. This model may serve as a basic building block of brain functions involving associative memory.

Multiscale memory and bioelectric error correction in the cytoplasm-cytoskeleton-membrane system.

A fundamental aspect of life is the modification of anatomy, physiology, and behavior in the face of changing conditions. This is especially illustrated by the adaptive regulation of growth and form that underlies the ability of most organisms-from single cells to complex large metazoa-to develop, remodel, and regenerate to specific anatomical patterns. What is the relationship of the genome and other cellular components to the robust computations that underlie this remarkable pattern homeostasis? Here we examine the role of constraints defined at the cellular level, especially endogenous bioelectricity, in generating and propagating biological information. We review evidence that the genome is only one of several multi-generational biological memories. Focusing on the cell membrane and cytoplasm, which is physically continuous across all of life in evolutionary timeframes, we characterize the environment as an interstitial space through which messages are passed via bioelectric and biochemical codes. We argue that biological memory is a fundamental phenomenon that cannot be understood at any one scale, and suggest that functional studies of information propagated in non-genomic cellular structures will not only strongly impact evolutionary developmental biology, but will also have implications for regenerative medicine and synthetic bioengineering. WIREs Syst Biol Med 2018, 10:e1410. doi: 10.1002/wsbm.1410 This article is categorized under: Developmental Biology > Stem Cell Biology and Regeneration Physiology > Physiology of Model Organisms Models of Systems Properties and Processes > Cellular Models.

Race in an epigenetic time: thinking biology in the plural.

The notion that biological memories of environmental experiences can be embedded in the human genome and even transmitted transgenerationally is increasingly relevant in the postgenomic world, particularly in molecular epigenetics, where the genome is conceptualized as porous to environmental signals. In this article I discuss the current rethinking of race in epigenetic rather than genetic terms, emphasizing some of its paradoxical implications, especially for public policy. I claim in particular that: (i) if sociologists want to investigate race in a postgenomic world they should pay more attention to this novel plastic and biosocial view of race; and (ii) there are no reasons to believe that an epigenetic view will extinguish race, or that soft-inheritance claims will produce a less exclusionary discourse than genetics (hard heredity). Quite the opposite, the ground for a re-racialization of social debates and the reinforcement of biological boundaries between groups are highlighted in the article.

Complex synapses as efficient memory systems

The molecular machinery underlying memory consolidation at the level of synaptic connections is believed to employ a complex network of highly diverse biochemical processes that operate on a wide range of different timescales. An appropriate theoretical framework could help us identify their computational roles and understand how these intricate networks of interactions support synaptic memory formation and maintenance. Here we construct a broad class of synaptic models that can efficiently harness biological complexity to store and preserve a huge number of memories, vastly outperforming other synaptic models of memory. The number of storable memories grows almost linearly with the number of synapses, which constitutes a substantial improvement over the square root scaling of previous models [1,2], especially when large neural systems are considered. This improvement is obtained without significantly reducing the initial memory strength, which still scales approximately like the square root of the number of synapses. This is achieved by combining together multiple dynamical processes that operate on different timescales, to ensure the memory strength decays as slowly as the inverse square root of the age of the corresponding synaptic modification. Memories are initially stored in fast variables and then progressively transferred to slower ones. Importantly, in our case the interactions between fast and slow variables are bidirectional, in contrast to the unidirectional cascades of previous models. The proposed models are robust to perturbations of parameters and can capture several properties of biological memories, which include delayed expression of synaptic potentiation and depression, synaptic metaplasticity, and spacing effects. We discuss predictions for the autocorrelation function of the synaptic efficacy that can be tested in plasticity experiments involving long sequences of synaptic modifications.

Documenting, storing, and executing models in Ecology: A conceptual framework and real implementation in a global change monitoring program

Many of the best practices concerning the development of ecological models or analytic techniques published in the scientific literature are not fully available to modelers but rather are stored in scientists' digital or biological memories. We propose that it is time to address the problem of storing, documenting, and executing ecological models and analytical procedures. In this paper, we propose a conceptual framework to design and implement a web application that will help to meet this challenge. This tool will foster cooperation among scientists, enhancing the creation of relevant knowledge that could be transferred to environmental managers. We have implemented this conceptual framework in a tool called ModeleR. This is being used to document, share, and execute more than 200 models and analytical processes associated with a global change monitoring program that is being undertaken in the Sierra Nevada Mountains (south Spain). ModeleR uses the concept of scientific workflow to connect and execute different types of models and analytical processes. Finally, we have envisioned the creation of a federation of model repositories where models documented within a local repository could be linked and even executed by other researchers.

Biological Memories and Agents as Quantum Collectives

Quantum mechanical models have been proposed for biological processes and for cognition and decision in domains that appear to be beyond the de Broglie wavelength. The basis of such quantum behavior is seen variously as quantum fields and virtual and entangled particles, and the determination that the behavior is quantum is made on coherence, order and interference effects, and non-local behavior. This paper proposes that biological memories and cognitive agents are collectives of quantum objects. Statistical and informational properties of the collectives that need to be taken into consideration are identified. Issues related to mapping of collectives into various energy states and resistance to noise are examined. NeuroQuantology | September 2013 | Volume 11 | Issue 3 | Page 391-398

Micro UHMW‐PE column array molded by the utilization of PCB as mold insert

PurposeThe purpose of this paper is to present a method for ultrasonically molding polymer powder in a micro plastic part mold. In the method, a printed circuit board (PCB) in which micro‐hole arrays are drilled is used as a micro cavity insert. With the utilization of ultrasonic vibration, the polymer powder, which is prefilled and compacted in a micro cavity, mutually generates great sliding friction heat so as to be rapidly plasticized and molded.Design/methodology/approachMicro carbide drill bits of which the diameters are 100.0 μm, 150.0 μm and 200.0 μm, respectively, are used for drilling the PCB to form a micro‐hole array insert. Next, two kinds of various ultra‐high molecule weight polyethylene (UHMW‐PE) powder with various grain diameters are directly filled into a charging barrel and a mold cavity with the micro‐hole array insert. Proper process parameters are set on ultrasonic plasticizing and molding equipment so that a molding test can be performed. The melt of UHMW‐PE can be rapidly filled into the cavity. Finally, micro‐column array plastic parts are successfully prepared.FindingsThe micro‐hole array PCB is a mold insert which is quite applicable for the ultrasonic molding of the powder in the mold. When a molding material is the coarse UHMW‐PE powder with the grain diameter of about 350 μm, the diameter replication rates of the micro‐column array plastic parts become good in order with the increased micro‐hole diameter of the PCB. When the fine UHMW‐PE powder with the grain diameter of about 80 μm is adopted, the diameter replication rates of the micro‐column array plastic parts become good in order with the decreased micro‐hole diameter of the PCB.Originality/valueIn this paper, the micro‐column array plastic parts with good replicability are successfully prepared by a technique for ultrasonically plasticizing and molding in the cavity. The technique can be applied to the fields of medical treatment, communication, optics, chemistry and so on, such as biological micro needle arrays, micro biological chips, optical memories, and micro chemical reaction chips.

Short-Term Memory to Long-Term Memory Transition in a Nanoscale Memristor

"Memory" is an essential building block in learning and decision-making in biological systems. Unlike modern semiconductor memory devices, needless to say, human memory is by no means eternal. Yet, forgetfulness is not always a disadvantage since it releases memory storage for more important or more frequently accessed pieces of information and is thought to be necessary for individuals to adapt to new environments. Eventually, only memories that are of significance are transformed from short-term memory into long-term memory through repeated stimulation. In this study, we show experimentally that the retention loss in a nanoscale memristor device bears striking resemblance to memory loss in biological systems. By stimulating the memristor with repeated voltage pulses, we observe an effect analogous to memory transition in biological systems with much improved retention time accompanied by additional structural changes in the memristor. We verify that not only the shape or the total number of stimuli is influential, but also the time interval between stimulation pulses (i.e., the stimulation rate) plays a crucial role in determining the effectiveness of the transition. The memory enhancement and transition of the memristor device was explained from the microscopic picture of impurity redistribution and can be qualitatively described by the same equations governing biological memories.

Reduced order multiport parallel and multidirectional neural associative memories

This paper proposes multiport parallel and multidirectional intraconnected associative memories of outer product type with reduced interconnections. Some new reduced order memory architectures such as k-directional and k-port parallel memories are suggested. These architectures are, also, very suitable for implementation of spatio-temporal sequences and multiassociative memories. It is shown that in the proposed memory architectures, a substational reduction in interconnections is achieved if the actual length of original N-bit long vectors is subdivided into k sublengths. Using these sublengths, submemory matrices, T ( s ) or W ( s ), are computed, which are then intraconnected to form k-port parallel or k-directional memories. The subdivisions of N-bit long vectors into k sublengths save ((k-1) x 100) / k % of interconnections. It is shown, by means of an example, that more than 80% reduction in interconnections is achieved. Minimum limit in bits on k as well as maximum limit on subdivisions in k is determined. The topologies of reduced interconnectivity developed in this paper are symmetric in structure and can be used to scale up to larger systems. The underlying principal of construction, storage and retrieval processes of such associative memories has been analyzed. The effect of complexity of different levels of reduced interconnectivity on the quality of retrieval, signal to noise ratio, and storage capacity has been investigated. The model possesses analogies to biological neural structures and digital parallel port memories commonly used in parallel and multiprocessing systems.

From creation to consolidation: a novel framework for memory processing.

Long after playing a game of squash or reading this essay, your memory for playing and reading continues to be processed by your brain. These “offline” processes improve your game and your understanding of this essay, and more generally, enhance adaptive behavior. Yet progress in understanding how the brain regulates the offline processing of memories has been hampered by the absence of robust models for interpreting diverse, and often contradictory, experimental results. In the last 20 years, highly fertile quantitative models across the biological spectrum from the molecular to the behavioral have proved critical in advancing our understanding of memory encoding (e.g., [1–5]). But these models have focused upon the exact moment a memory is formed; while our ability to recall an event is dictated, at least in part, by events that precede and follow the encoding of a new memory. The critical role that events following memory encoding play in determining subsequent recall have been recognized for at least the past 100 years [6]. Yet few, if any, models have been formulated for these “offline” processes that produce qualitative and quantitative changes in a memory during consolidation (Box 1). Our attempts to understand these mysterious processes have generated a purely descriptive set of observations. Although these observations have provided critical glimpses into offline memory processing, they have also produced unresolved contradictions between some of the most fundamental and critical sets of observations (for reviews, see [7–11]). Box 1. Memory Consolidation A memory passes through at least three key milestones in its development: initially it is encoded, then it is consolidated, and finally it is retrieved. During consolidation a memory can undergo both quantitative and qualitative changes. A memory may be enhanced, demonstrated by a quantitative increase in performance, or it may be stabilized, demonstrated by becoming quantitatively less susceptible to interference [10,46,47]. A memory can also undergo qualitative changes: there can be a shift in the strategy used to solve a problem or the emergence of awareness for what had earlier been learned [49,50]. Although there is a rich diversity in the behavioral expression of consolidation, each of these examples may rely upon the same underlying computation (see main text). Consolidation is measured as a change in performance between testing and retesting [46,47]. Contrasting final performance at retesting against an initial baseline provides a direct measure of “offline” performance changes that occur during consolidation. For example, one set of observations suggests that consolidation may occur over any time interval, whereas another body of data suggests that these processes require sleep [6,8]. Clearly, both cannot be true. Resolving the inherent conflict between these perspectives strikes at the very heart of how biological mechanisms process memories after their initial encoding. Making sense of what threatens to become an avalanche of disconnected and incoherent empirical findings may require novel theories that can simultaneously reconcile apparently inconsistent observations and provide a fertile, hypothesis-driven framework for future work. Here, drawing upon examples mainly from the processing of motor skill memories, I take the first tentative steps toward assembling such a framework.

Neural memories and search engines

In this article, we show the existence of a formal convergence between the matrix models of biological memories and the vector space models designed to extract information from large collections of documents. We first show that, formally, the term-by-document matrix (a mathematical representation of a set of codified documents) can be interpreted as an associative memory. In this framework, the dimensionality reduction of the term-by-document matrices produced by the latent semantic analysis (LSA) has a common factor with the matrix biological memories. This factor consists in the generation of a statistical ‘conceptualisation’ of data using little dispersed weighted averages. Then, we present a class of matrix memory that built up thematic blocks using multiplicative contexts. The thematic memories define modular networks that can be acceded using contexts as passwords. This mathematical structure emphasises the contacts between LSA and matrix memory models and invites to interpret LSA, and similar procedures, as a reverse engineering applied on context-deprived cognitive products, or on biological objects (e.g. genomes) selected during large evolutionary processes.

Cortical memory dynamics.

Biological memories have a number of unique features, including (1) hierarchical, reciprocally interacting layers, (2) lateral inhibitory interactions within layers, and (3) Hebbian synaptic modifications. We incorporate these key features into a mathematical and computational model in which we derive and study Hebbian learning dynamics and recall dynamics. Introducing the construct of a feasible memory (a memory that formally responds correctly to a specified collection of noisy cues that are known in advance), we study stability and convergence of the two kinds of dynamics by both analytical and computational methods. A conservation law for memory feasibility under Hebbian dynamics is derived. An infomax net is one where the synaptic weights resolve the most uncertainty about a neural input based on knowledge of the output. The infomax notion is described and is used to grade memories and memory performance. We characterize the recall dynamics of the most favorable solutions from an infomax perspective. This characterization includes the dynamical behavior when the net is presented with external stimuli (noisy cues) and a description of the accuracy of recall. The observed richness of dynamical behavior, such as its initial state sensitivity, provides some hints for possible biological parallels to this model.

Simple biological neural networks and stability criteria for equilibrium memories

The relationship between properties of recently very intensively studied artificial neural networks and real biological neural networks is discussed. It is indicated that there are many deep differences, so far. One is forced to model such systems on the basis of dynamical system theory. Besides it is clear that one has to think about stable steady states instead of equilibrium ones as appropriate memories. The necessary conditions of stability of equilibrium memory states are derived. This is promising to be of great practical meaning in applications of Hopfield-like nets of this type. On the other hand it seems that in biological neural nets memories would work on the basis of more complex attractors. Indications are made that most probably are strange attractors.

Internal representations for associative memory

We describe a class of feed forward neural network models for associative content addressable memory (ACAM) which utilize sparse internal representations for stored data. In addition to the input and output layers, our networks incorporate an intermediate processing layer which serves to label each stored memory and to perform error correction and association. We study two classes of internal label representations: the unary representation and various sparse, distributed representations. Finally, we consider storage of sparse data and sparsification of data. These models are found to have advantages in terms of storage capacity, hardware efficiency, and recall reliability when compared to the Hopfield model, and to possess analogies to both biological neural networks and standard digital computer memories.