Firing Rate Models as Associative Memory: Synaptic Design for Robust Retrieval.

Event Abstract Back to Event Medial temporal lobe substructures differentially contribute to processing within- and between-domain associative recognition memory for semantic and non-semantic stimuli Marshall A. Dalton1, 2*, Michael Hornberger1, 2 and Olivier Piguet1, 2 1 Neuroscience Research Australia , Australia 2 School of Medical Sciences, University of New South Wales, Australia Introduction The domain dichotomy theory posits that medial temporal lobe (MTL) substructures differentially contribute to different types of associative memory processing with the hippocampus implicated in the integration of information from different perceptual or conceptual domains (between-domain [BD] memory processing) and the perirhinal cortex involved in within-domain (WD) memory processing. The aim of this study was to determine whether the neural correlates of recognition memory on WD and BD associative memory tasks differ when processing semantic and non-semantic stimuli. Methods Twenty healthy adults participated in an fMRI experiment comprising three associative memory tasks: (i) a WD task using non-semantic stimuli, (ii) a WD task using compound words, (iii) a BD task using a word-location paradigm. Results WD associative recognition memory for non-semantic stimuli correlated with an increased BOLD signal in the right perirhinal cortex. WD associative recognition memory for semantic stimuli correlated with an increased BOLD signal in the left anterior hippocampus. BD associative recognition memory correlated with an increased BOLD signal in the left posterior hippocampus. Conclusions Results of this study support the theory that MTL substructures differentially contribute to different types of associative memory processing and in addition show that MTL substructures also differentially contribute to associative memory processing of semantic and non-semantic stimuli. These results have important implications for theoretical models of associative and episodic memory processing in the human MTL. Acknowledgements This work was supported by the Australian Research Council [DP110104202 to M.H.] and the National Health and Medical Research Council of Australia [APP1022684 to O.P.]. M.A.D. is supported by an Australian Rotary Health award. Keywords: medial temporal lobe, Hippocampus, perirhinal cortex, Associative Memory, episodic memory Conference: ACNS-2012 Australasian Cognitive Neuroscience Conference, Brisbane, Australia, 29 Nov - 2 Dec, 2012. Presentation Type: Oral Presentation Topic: Memory Citation: Dalton MA, Hornberger M and Piguet O (2012). Medial temporal lobe substructures differentially contribute to processing within- and between-domain associative recognition memory for semantic and non-semantic stimuli. Conference Abstract: ACNS-2012 Australasian Cognitive Neuroscience Conference. doi: 10.3389/conf.fnhum.2012.208.00136 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 25 Oct 2012; Published Online: 17 Nov 2012. * Correspondence: Mr. Marshall A Dalton, Neuroscience Research Australia, Sydney, Australia, m.dalton@neura.edu.au Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Marshall A Dalton Michael Hornberger Olivier Piguet Google Marshall A Dalton Michael Hornberger Olivier Piguet Google Scholar Marshall A Dalton Michael Hornberger Olivier Piguet PubMed Marshall A Dalton Michael Hornberger Olivier Piguet Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

A firing rate (FR) model for a population of adaptive integrate-and-fire (IF) neurons has been proposed. Unlike known FR models, it describes more precisely the unsteady firing regimes and takes into account the effect of slow potassium currents of spike-time adaptation. Approximations of the adaptive channel conductances are rewritten from voltage-dependent to spike-dependent and then to rate-dependent ones. The proposed FR model is compared to the very detailed population model, namely, the conductance-based refractory density model. The comparison of this model with the full RD model shows the coincidence of the first peak of activity after the start of stimulation as well as the stationary state. As an example of the simulation of coupled adaptive neuronal populations, a ring model has been constructed, which reproduces a visual illusion named tilt after-effect. The FR model is recommended for the mathematical analysis of neuronal population activity as well as for computationally expensive large-scale simulations.

Synchronization plays important role in generation of brain activity patterns. Experimental data show that neurons demonstrate more reproducible activity for noise-like input than for constant current injection, and that effect can not be reproduced by standard oversimplified Firing-Rate (FR) models. The paper proposes a modification of FR model which reproduces these kinds of activity. The FR model approximates the firing rate of an infinite number of leaky integrate-and-fire neurons, considered as a population, and in contrary to conventional models it accounts for not only a steady-state firing regime but a fast rising excitation as well. Comparison of our simulations with the experimental data shows that the synchronous firing of the neuronal population strongly depends on the synchrony of neuronal states just before spiking. This effect is reproduced by the proposed FR model in contrary to the conventional FR models and is in agreement with the direct Monte-Carlo simulation of individual neurons.

Event Abstract Back to Event Modeling firing-rate dynamics: From spiking to firing-rate networks Evan S. Schaffer1* and L. F. Abbott1 1 Columbia University, United States Firing-rate models provide an attractive approach for studying large neural networks because they can be simulated rapidly and are amenable to mathematical analysis. Traditional firing-rate models have the obvious shortcoming of using a single time constant (typically a membrane or synaptic time constant) to describe all changes in rate, and they require neurons in a network to fire asynchronously. This is likely to be violated in many cases; in fact, transient synchronization of subgroups of neurons may be an important mechanism for generating rapid behavioral responses. To address this issue without losing the advantages associated with a simple firing-rate description, we have developed a form of firing-rate model based on an approximate Fokker-Planck analysis. A Fokker-Planck equation can be used to describe the distribution of membrane potentials for a population of neurons receiving noisy input. However, most methods for approximating solutions to this type of equation lead to considerably more complex equations than are practical for large networks. For example, there is no closed-form solution describing a population of Integrate-and-Fire (IAF) neurons receiving arbitrary time-varying input. A linear approximation to the response can be calculated, yielding impressively high accuracy, but it involves cumbersome equations (e.g. Brunel & Hakim, 1999; Mattia & Del Giudice, 2002; Ostojic et al., 2009). We show here that in a variant of this model, the Quadratic IAF, the fully nonlinear rate response can be approximated in a surprisingly simple form. Importantly, this approximate solution makes no assumptions about the shape, amplitude, or continuity of the input current. With an understanding of how dynamic external inputs drive firing rates for both asynchronous and synchronous populations of neurons, we study how units described in this way can be linked to describe the firing-rate dynamics of spiking networks with various patterns of synaptic connectivity. We find that the novel firing-rate model captures the time-varying firing-rates of the spiking network across a wide range of parameters. This holds equally well in parameter ranges where the asynchronous state is stable, and where highly synchronized firing occurs. Furthermore, the model also reproduces the dynamics of transient synchronization, which can be quite complicated. Finally, we show that the rich firing dynamics of a network of both excitatory and inhibitory neurons can be well approximated by a coupled E-I rate network. The simplicity of the model we have derived makes it highly amenable to use as the basis for network models. This will hopefully make tractable the study of the dynamics of large networks. Conference: Computational and Systems Neuroscience 2010, Salt Lake City, UT, United States, 25 Feb - 2 Mar, 2010. Presentation Type: Poster Presentation Topic: Poster session II Citation: Schaffer ES and Abbott LF (2010). Modeling firing-rate dynamics: From spiking to firing-rate networks. Front. Neurosci. Conference Abstract: Computational and Systems Neuroscience 2010. doi: 10.3389/conf.fnins.2010.03.00269 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 05 Mar 2010; Published Online: 05 Mar 2010. * Correspondence: Evan S Schaffer, Columbia University, New York, United States, ess2129@columbia.edu Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Evan S Schaffer L. F Abbott Google Evan S Schaffer L. F Abbott Google Scholar Evan S Schaffer L. F Abbott PubMed Evan S Schaffer L. F Abbott Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Gamma oscillations are widely observed in the mammalian brain and are important markers for cognition and attention [1,2]. In CA1 of the hippocampus of freely moving rats, power in one of two distinct oscillatory bands in the gamma regime (fast gamma and slow gamma) is predominantly present at a given moment of time [3]. Here, we demonstrate that models of networks with competing interneuron populations with different post-synaptic effects can create distinct oscillatory regimes that mimic the observed oscillations of CA1. Our network formulation reflects the following facts: 1) The duration of post-synaptic effect of an interneuron strongly influences the frequency in biophysical models of gamma oscillations [4]. 2) The primary CA1 inputs from CA3 and the entorhinal cortex (EC) preferentially innervate interneurons of different subtype with different post-synaptic durations [5,6]. We show that a firing rate model with competing interneuron populations with different post-synaptic time-constants is sufficient to generate slow and fast gamma oscillations. We conclude that mutual inhibition between the modeled interneuron populations permits switching in a bistable regime between distinct fast and slow gamma states. We also find similar behavior in spike-based network models. Our models explicitly predict the following about CA1: 1) Different interneurons innervated by different upstream regions phase-lock to different gamma states. 2) One population of interneurons is silenced, and another is active during fast and slow gamma events. 3) Mutual inhibition between interneuron populations is necessary for spontaneous switching of gamma state. Using experimental electrophysiological data from awake behaving rodents, we find interneurons that satisfy conditions 1 and 2, and we show putative 'fast' and 'slow' gamma interneurons categorized by their tendency to fire and phase-lock with oscillatory events as measured by a nearby local field potential. Our 3-population firing rate model is schematized in Figure ​Figure1A.1A. The dynamic variables are synaptic currents of an excitatory, fast inhibitory (IF) and slow inhibitory (Is) population; the firing rates are instantaneous functions of total input current. Fast excitation that interacts with inhibitory subpopulations supports oscillations. This interaction engages either one or both inhibitory subpopulations depending on IS - IF connectivity and input balance (Example in Figure ​Figure1B).1B). This network oscillates at biophysically realistic frequencies given biophysically realistic network parameters. The fast inhibitory population, IF and slow inhibitory population, IS have post-synaptic time-constants of 5ms and 15ms, respectively. These roughly capture the diversity of post-synaptic inhibitory current time-courses of interneurons of different subtypes measured in CA1 [6]. Our firing rate model demonstrates that with sufficient mutual inhibition between inhibitory populations, the oscillating network bifurcates into two stable regimes that oscillate at roughly the same frequencies as the observed fast and slow gamma oscillations [3,7]. Figure 1 A. Connectivity scheme for firing-rate two-gamma model. CA3 and EC denote inputs, E denotes excitatory population, and IF and IS denote the interneuron populations with brief and long post-synaptic effects, respectively. B. Graph shows the oscillation ... Previous experimental work suggests these two gamma oscillations reflect different information processing modes in the learning and memory system [7]. Our models provide a mechanistic understanding of these modes and posit a new oscillatory role for distinct interneurons in CA1. Moreover, our models describe general oscillatory behavior in networks with distinct interneuron populations.

The macroscopic dynamics of large populations of neurons can be mathematically analyzed using low-dimensional firing-rate or neural-mass models. However, these models fail to capture spike synchronization effects and nonstationary responses of the population activity to rapidly changing stimuli. Here we derive low-dimensional firing-rate models for homogeneous populations of neurons modeled as time-dependent renewal processes. The class of renewal neurons includes integrate-and-fire models driven by white noise and has been frequently used to model neuronal refractoriness and spike synchronization dynamics. The derivation is based on an eigenmode expansion of the associated refractory density equation, which generalizes previous spectral methods for Fokker-Planck equations to arbitrary renewal models. We find a simple relation between the eigenvalues characterizing the timescales of the firing rate dynamics and the Laplace transform of the interspike interval density, for which explicit expressions are available for many renewal models. Retaining only the first eigenmode already yields a reliable low-dimensional approximation of the firing-rate dynamics that captures spike synchronization effects and fast transient dynamics at stimulus onset. We explicitly demonstrate the validity of our model for a large homogeneous population of Poisson neurons with absolute refractoriness and other renewal models that admit an explicit analytical calculation of the eigenvalues. The eigenmode expansion presented here provides a systematic framework for alternative firing-rate models in computational neuroscience based on spiking neuron dynamics with refractoriness.

Firing rate models are powerful phenomenological descriptions of the collective activity of large networks of spiking neurons. A prominent feature of the dynamics of large neuronal networks are the synchrony-driven collective oscillations generated by the interplay between synaptic delays and recurrent coupling. This thesis investigates the emergence of delay-induced oscillations in networks of heterogeneous spiking neurons. The analysis is carried out by means of a novel firing rate model that exactly describes the average firing rate and membrane potential dynamics of a network of model spiking neurons (of quadratic integrate-and-fire type). We consider networks with three forms of synaptic delays: i) Fixed delays, ii) Instantaneous rise and single-exponential decay synaptic kinetics, and iii) a combination of the two. For the three forms of synaptic coupling, we obtain phase diagrams analytically to a large extent, which determine the presence of various oscillatory states in regions of the space of parameters of the network. These states include nontrivial synchronous regimes (with quasiperiodic and chaotic dynamics) which are analyzed by performing extensive numerical simulations in the original network of spiking neurons. Finally, comparisons with the traditional firing rate models vastly used in computational neuroscience are performed. In the limit of slow synaptic kinetics and vanishing delays, traditional firing rate models adequately describe the dynamics of the network of spiking neurons. However, for other forms of synaptic delays, the traditional rate equations only provide a faithful description of the network dynamics in the case of strong heterogeneity and inhibitory coupling

Associative learning and memory are fundamental behavioral processes through which organisms adapt to complex environments. Associative memory involves long-lasting changes in synaptic plasticity. Dendritic spines are tiny protrusions from the dendritic shaft of principal neurons, providing the structural basis for synaptic plasticity and brain networks in response to external stimuli. Mounting evidence indicates that dendritic spine dynamics are crucial in different associative memory phases, including acquisition, consolidation, and reconsolidation. Causally bridging dendritic spine dynamics and associative memory is still limited by the suitable tools to measure and control spine dynamics in vivo under behaviorally relevant conditions. Here, we review data providing evidence for the remodeling of dendritic spines during associative memory processing and outline open questions.

The firing rate model in the form of nonlinear integrodifferential equations can characterize spatiotemporal patterns of a continuum neural field. These patterns are associated with a wide range of neurobiological phenomena, such as persistent activity and propagating waves in neural networks. To understand the substrates of neural circuitry that supports the localized stationary patterns, we study the existence of multi-bump pulse solutions of an integral equation that is the equilibrium equation of the firing rate model. If the integral coupling function, which describes the spatial connection among the network of neurons, is even and its positive half solves a second order linear ordinary differential equation (ODE), then the multi-bump pulse solutions of the integral equation are homoclinic solutions of a reversible fourth order ODE. It was known previously that the corresponding ODEs are conservative for a class of oscillatory and decaying coupling functions [1]. We show that for the Amari-type firing rate model, which assumes the Heaviside firing rate function and Mexican hat coupling functions that decay exponentially, the corresponding ODEs are also conservative. Then, by analyzing the configurations of stable and unstable manifolds (see the figure below) in the corresponding reversible, conservative ODE system, we establish the existence of the Smale horseshoe for an open set of model parameters such as the decay rate of the coupling and the threshold of the Heaviside firing rate function. Consequently, even for the Amari-type firing rate model there are countably many symmetric and asymmetric multi-bump stationary pulse solutions as well as spatially chaotic stationary solutions. Furthermore, the robustness of the Smale horseshoe implies that similar solutions also exist for nonsaturating piecewise-linear firing rate functions with small gains and for smooth firing rate functions that are close to the Heaviside or piecewise-linear case. Figure 1 Multi-bump pulse solutions correspond to intersections between the stable and unstable manifolds of the equilibrium state in the fourth order reversible ODE system. Since the ODE system is conservative, the intersections between these manifolds can be ...

Prior studies suggested that the transcription factor ATF4 negatively regulates synaptic plastic and memory. By contrast, we provide evidence from direct in vitro and in vivo knockdown of ATF4 in rodent hippocampal neurons and from ATF4-null mice that implicate ATF4 as essential for normal synaptic plasticity and memory. In particular, hippocampal ATF4 downregulation produces deficits in long-term spatial memory and behavioral flexibility without affecting associative memory or anxiety-like behavior. ATF4 knockdown or loss also causes profound impairment of both long-term potentiation (LTP) and long-term depression (LTD) as well as decreased glutamatergic function. We conclude that ATF4 is a key regulator of the physiological state necessary for neuronal plasticity and memory.

Bidirectional Plasticity Gated by Hyperpolarization Controls the Gain of Postsynaptic Firing Responses at Central Vestibular Nerve Synapses

Chemical and electrical synapses shape the dynamics of neuronal networks. Numerous theoretical studies have investigated how each of these types of synapses contributes to the generation of neuronal oscillations, but their combined effect is less understood. This limitation is further magnified by the impossibility of traditional neuronal mean-field models-also known as firing rate models or firing rate equations-to account for electrical synapses. Here, we introduce a firing rate model that exactly describes the mean-field dynamics of heterogeneous populations of quadratic integrate-and-fire (QIF) neurons with both chemical and electrical synapses. The mathematical analysis of the firing rate model reveals a well-established bifurcation scenario for networks with chemical synapses, characterized by a codimension-2 cusp point and persistent states for strong recurrent excitatory coupling. The inclusion of electrical coupling generally implies neuronal synchrony by virtue of a supercritical Hopf bifurcation. This transforms the cusp scenario into a bifurcation scenario characterized by three codimension-2 points (cusp, Takens-Bogdanov, and saddle-node separatrix loop), which greatly reduces the possibility for persistent states. This is generic for heterogeneous QIF networks with both chemical and electrical couplings. Our results agree with several numerical studies on the dynamics of large networks of heterogeneous spiking neurons with electrical and chemical couplings.

A new content addressable associative memory structure, associative cellular memory, is proposed. It is composed of simple associative memory cells that are embedded in a regular low dimensional grid and interconnected locally within a neighborhood of a limited distance. This paper describes the ideas behind associative cellular memory and gives an example in the field of pattern recognition. The associative cellular memory, the Kanerva's sparse distributed memory, and the Hopfield network are compared in terms of memory capacity, run-time efficiency and information density. >

When observers view for extended time an ambiguous visual scene with two or more different interpretations they report switching between different perceptions. We focus on a classical paradigmatic stimulus, the visual plaids, consisting of two superimposed drifting gratings with transparent intersections [1,2]. For visual plaids, tristable perception is experienced: one coherent percept (the gratings move together as a single pattern) and two transparent percepts (the gratings slide across one another) with alternating depth order [3]. In order to decipher the complex mechanisms of tristable perception, we gathered a large amount of psychophysical data on tristable plaids and developed a neural network, firing rate model of interaction between neural populations that could account for the experimental results. Nine subjects reported continuously the three possible percepts for 3 sessions of 10 stimuli each (3 minutes per stimulus). The angle between the vectors normal to gratings was equal to 80, 100 or 120 degrees. We collected enough percepts to compute statistics for each subject and parameter without collapsing data [4]. As opposed to bistable stimuli where the only possibility is the alternation between the two percepts, in the tristable case the results show that the next percept probability depends on the previous percepts. Indeed, the sequence of perceptual switches confirms that switches between two transparency states are typically interleaved by a coherent percept, especially for values of the angle equal to 80 and 100 [3,4]. Moreover, by examining triplets consisting of two transparent percepts interleaved by a coherent one, we observed that the probability of the two transparent percepts in the triplet having the same depth pattern decreases as the duration of the coherent percept shortens. These trends suggest that adaptation is implicated in perceptual alternations. For bistable alternations correlations are absent or insignificant. We propose inhibition-based competition along with adaptation and noise in a multi-state framework as plausible mechanisms for the dynamics of perceptual switching. Our model is a firing rate model consisting of three mutually coupled populations of cells, each one encoding a different percept. It is based on the firing rate models for alternations during perceptual bistability [5]. We can explain the dependence in perceptual history by introducing an inhibition imbalance in the interactions between neural populations (the two transparent percepts inhibit each other more strongly than they inhibit the coherent state, making the latter more dominant and more likely to switch to). Adjusting the relative strength of adaptation and noise we can account for the dominance duration distributions and the switching probability between depth percepts as a function of the coherent percept duration. Finally, we consider other possible architectures for the model and we show that a non-hierarchical architecture where motion is encoded together with depth fits better with the experimental results.

Editor's evaluation: Nonlinear transient amplification in recurrent neural networks with short-term plasticity