Spike Arrival Research Articles

Reducing ANN-SNN Conversion Error through Residual Membrane Potential

Spiking Neural Networks (SNNs) have received extensive academic attention due to the unique properties of low power consumption and high-speed computing on neuromorphic chips. Among various training methods of SNNs, ANN-SNN conversion has shown the equivalent level of performance as ANNs on large-scale datasets. However, unevenness error, which refers to the deviation caused by different temporal sequences of spike arrival on activation layers, has not been effectively resolved and seriously suffers the performance of SNNs under the condition of short time-steps. In this paper, we make a detailed analysis of unevenness error and divide it into four categories. We point out that the case of the ANN output being zero while the SNN output being larger than zero accounts for the largest percentage. Based on this, we theoretically prove the sufficient and necessary conditions of this case and propose an optimization strategy based on residual membrane potential to reduce unevenness error. The experimental results show that the proposed method achieves state-of-the-art performance on CIFAR-10, CIFAR-100, and ImageNet datasets. For example, we reach top-1 accuracy of 64.32% on ImageNet with 10-steps. To the best of our knowledge, this is the first time ANN-SNN conversion can simultaneously achieve high accuracy and ultra-low-latency on the complex dataset. Code is available at https://github.com/hzc1208/ANN2SNN_SRP.

An ensemble unsupervised spiking neural network for objective recognition

It is now known that the spiking neuron is a basic unit of spiking neural networks (SNNs). Spiking neurons modulate the nervous cells via receiving external incentives, generation of action potential and firing spikes. The SNNs usually used for pattern recognition tasks or complex computation depending on the brain-like characteristic. Although the SNNs have no advantages comparing with the deep neural networks in terms of classification accuracy, the SNNs have more characteristics of biological neurons. In this paper, a hierarchical SNN, comprising convolutional and pooling layers, is designed. The proposed SNN consists of excitatory and inhibitory neurons based on the mechanism of the primate brain. A temporal coding (rank order) manner is used to encode the input patterns. It depends on the rank of the spike arrival on post synapses to establish the priority of input spikes for a particular pattern. The spike-timing-dependent plasticity (STDP) learning rule is used in convolutional layers to extract visual features in an unsupervised learning manner. During the classification stage, a lateral inhibition mechanism is used to prevent the non-firing neurons and produce distinguishable results. In order to improve the performance of our SNN, an ensemble SNN architecture using the voting method is proposed, and transfer learning is used to avoid re-training the SNN when solving the different tasks. The hand-written digits classification task on MNIST, CIFAR-10, and BreaKHis databases are used to verify the performance of the proposed SNN. Experimental results show that by using the ensemble architecture and transfer learning, the classification accuracy of 99.27% for the MNIST database, overall accuracy is 93% for the CIFAR-10 database, and overall accuracy is 96.97% for BreaKHis database. In the meantime, this work achieves a better performance than the benchmarking approaches. Taken together, the results of our work suggest that the ensemble SNN architecture with transfer learning is key to improving the performance of the SNN.

Presynaptic Spike-Driven Spike Timing-Dependent Plasticity With Address Event Representation for Large-Scale Neuromorphic Systems

Learning plays an important role in the brain to make it adaptive to dynamical environments. This paper presents a presynaptic spike-driven spike timing-dependent plasticity (STDP) learning rule in the address domain for a neuromorphic architecture using a synaptic connectivity table in an external memory at a local routing node. We contribute two aspects to the implementation of the learning rule for extended large-scale neuromorphic systems. First, we reduced buffer sizes required for tracing a spike train which is required to pair all presynaptic and postsynaptic spike for an STDP time window. This method implements an exponential decay STDP function with two parameters: the latest timestamp and the synaptic modification rate at the latest timestamp. It reduces the required buffer size compared to previous works. Second, we resolve a lack of reverse lookup table issue with the presynaptic spike-driven algorithm. The proposed algorithm holds causal updates at postsynaptic spikes until a next presynaptic spike arrival. This approach removes the need of a reverse lookup table required at a postsynaptic spike. We show the implementation of the proposed algorithm in an FPGA device and validate it with a spiking neural network configuration. The experiment results show the proposed algorithm is comparable qualitatively with a conventional STDP learning rule.

Mapping spiking neural networks onto a manycore neuromorphic architecture

We present a compiler for Loihi, a novel manycore neuromorphic processor that features a programmable, on-chip learning engine for training and executing spiking neural networks (SNNs). An SNN is distinguished from other neural networks in that (1) its independent computing units, or "neurons", communicate with others only through spike messages; and (2) each neuron evaluates local learning rules, which are functions of spike arrival and departure timings, to modify its local state. The collective neuronal state dynamics of an SNN form a nonlinear dynamical system that can be cast as an unconventional model of computation. To realize such an SNN on Loihi requires each constituent neuron to locally store and independently update its own spike timing information. However, each Loihi core has limited resources for this purpose and these must be shared by neurons assigned to the same core. In this work, we present a compiler for Loihi that maps the neurons of an SNN onto and across Loihi's cores efficiently. We show that a poor neuron-to-core mapping can incur significant energy costs and address this with a greedy algorithm that compiles SNNs onto Loihi in a power-efficient manner. In so doing, we highlight the need for further development of compilers for this new, emerging class of architectures.

Local Spinal Cord Circuits and Bilateral Mauthner Cell Activity Function Together to Drive Alternative Startle Behaviors

The reticulospinal Mauthner cells (M-cells) of the startle circuit have been considered to be dedicated to one basic motor output and the C-type startle response in fish. The neural circuit underlying the C-start, a startle behavior in which the fish forms a "C"-shaped body bend has been described in depth in goldfish and zebrafish [1, 2] and is thought to occur in other species [3, 4]. However, previous research has shown that some species can perform a second type of startle called the S-start [5-7]. This startle response, in which the first movement creates an "S"-shaped body bend achieved with regional muscle activity on left and right sides, cannot be explained by M-cell circuit models. Here we use larval zebrafish to examine the S-start circuit. Since S-starts are elicited through tail stimulation [5-7] and ablating M-cells abolishes short-latency tail-elicited startles [8, 9], we hypothesized that M-cell activity was necessary for S-start generation. Our findings show that the M-cells fire simultaneously to generate the S-start. However, simultaneous M-cell spikes generated through direct current injection were not sufficient to generate S-starts. Through recordings of motoneurons, inhibitory interneurons, and sensory neurons, we uncover a mechanism for generating alternative startle behaviors; local sensory inputs drive inhibitory interneuron activity, which inhibits caudal motoneurons and pre-conditions their excitability prior to the arrival of M-cell spikes in the tail. We suggest that this motoneuron hyperpolarization can bias motor output to left or right sides, determining whether the fish performs a C-start or an S-start behavior.

A large fraction of neocortical myelin ensheathes axons of local inhibitory neurons.

Myelin is best known for its role in increasing the conduction velocity and metabolic efficiency of long-range excitatory axons. Accordingly, the myelin observed in neocortical gray matter is thought to mostly ensheath excitatory axons connecting to subcortical regions and distant cortical areas. Using independent analyses of light and electron microscopy data from mouse neocortex, we show that a surprisingly large fraction of cortical myelin (half the myelin in layer 2/3 and a quarter in layer 4) ensheathes axons of inhibitory neurons, specifically of parvalbumin-positive basket cells. This myelin differs significantly from that of excitatory axons in distribution and protein composition. Myelin on inhibitory axons is unlikely to meaningfully hasten the arrival of spikes at their pre-synaptic terminals, due to the patchy distribution and short path-lengths observed. Our results thus highlight the need for exploring alternative roles for myelin in neocortical circuits.

Synaptic and nonsynaptic plasticity approximating probabilistic inference.

Learning and memory operations in neural circuits are believed to involve molecular cascades of synaptic and nonsynaptic changes that lead to a diverse repertoire of dynamical phenomena at higher levels of processing. Hebbian and homeostatic plasticity, neuromodulation, and intrinsic excitability all conspire to form and maintain memories. But it is still unclear how these seemingly redundant mechanisms could jointly orchestrate learning in a more unified system. To this end, a Hebbian learning rule for spiking neurons inspired by Bayesian statistics is proposed. In this model, synaptic weights and intrinsic currents are adapted on-line upon arrival of single spikes, which initiate a cascade of temporally interacting memory traces that locally estimate probabilities associated with relative neuronal activation levels. Trace dynamics enable synaptic learning to readily demonstrate a spike-timing dependence, stably return to a set-point over long time scales, and remain competitive despite this stability. Beyond unsupervised learning, linking the traces with an external plasticity-modulating signal enables spike-based reinforcement learning. At the postsynaptic neuron, the traces are represented by an activity-dependent ion channel that is shown to regulate the input received by a postsynaptic cell and generate intrinsic graded persistent firing levels. We show how spike-based Hebbian-Bayesian learning can be performed in a simulated inference task using integrate-and-fire (IAF) neurons that are Poisson-firing and background-driven, similar to the preferred regime of cortical neurons. Our results support the view that neurons can represent information in the form of probability distributions, and that probabilistic inference could be a functional by-product of coupled synaptic and nonsynaptic mechanisms operating over several timescales. The model provides a biophysical realization of Bayesian computation by reconciling several observed neural phenomena whose functional effects are only partially understood in concert.

Proactive Workload Management in Hybrid Cloud Computing

The hindrances to the adoption of public cloud computing services include service reliability, data security and privacy, regulation compliant requirements, and so on. To address those concerns, we propose a hybrid cloud computing model which users may adopt as a viable and cost-saving methodology to make the best use of public cloud services along with their privately-owned (legacy) data centers. As the core of this hybrid cloud computing model, an intelligent workload factoring service is designed for proactive workload management. It enables federation between on- and off-premise infrastructures for hosting Internet-based applications, and the intelligence lies in the explicit segregation of base workload and flash crowd workload, the two naturally different components composing the application workload. The core technology of the intelligent workload factoring service is a fast frequent data item detection algorithm, which enables factoring incoming requests not only on volume but also on data content, upon a changing application data popularity. Through analysis and extensive evaluation with real-trace driven simulations and experiments on a hybrid testbed consisting of local computing platform and Amazon Cloud service platform, we showed that the proactive workload management technology can enable reliable workload prediction in the base workload zone (with simple statistical methods), achieve resource efficiency (e.g., 78% higher server capacity than that in base workload zone) and reduce data cache/replication overhead (up to two orders of magnitude) in the flash crowd workload zone, and react fast (with an X^2 speed-up factor) to the changing application data popularity upon the arrival of load spikes.

Axonal injuries and consequences to neuronal computation

Axonal injury and swelling is a common outcome of brain trauma and/or concussion. Recent experimentally observed axonal deformations induce irregularities in the axonal morphology and geometry [1,2]. Such irregularities, combined with nonlinear gating properties of ion channels, can significantly alter firing rate patterns between the initiation and arrival of spikes at pre-synaptic sites. As a consequence, information encoded in spike trains propagating in the axon can be jeopardized due to injury. We develop a computational framework to evaluate and distinguish between axonal injuries that lead to geometrical enlargements responsible for producing minor changes in propagation from those that result in critical phenomenon such as reflection or blockage of the original traveling spike train. We use a few geometrical parameters to model a prototypical axon enlargement and explore numerically the parameter space characterizing all possible propagation regimes and dynamics in an unmylienated action potential model. Figure ​Figure11 demonstrates an observed axon swelling [Figure ​[Figure1A]1A] a prototypical model [Figure ​[Figure1B]1B] along with its impact on a typical spike train [Figure ​[Figure1C].1C]. The geometry changes induced by the swelling can lead to significant change in the spike train dynamics and timing, thus jeopardizing faithful conductance of information. Figure 1 (A) Axonal swelling due to injury. (B) Model of enlargement. (C) Axonal geometry reshaping spike trains. Contrary to earlier notions that large diameter increases mostly lead to blocking, we demonstrate transmission is stable provided the geometrical changes occur in a slow (adiabatic) manner. As a consequence, swellings of lesser volumes but with abrupt increase in diameter can be more dangerous than those of larger volumes with adiabatic changes in geometry. Our method also identifies a narrow range of parameters leading to a reflection regime. The distinction between these three regimes can be evaluated by a simple analytical function of the geometrical parameters inferred through numerical simulations. Additionally, the effect on information encoding can be evaluated in such injured axons by performing a spike metric analysis [3]. Given that the statistics of both swelling size and frequency are known from recent experiments [1, 2, and references therein], our method allows or a prediction of the loss of neural computation due to traumatic brain injury and/or concussion.

Modeling fast stimulus–response association learning along the occipito-parieto-frontal pathway following rule instructions

On the basis of instructions, humans are able to set up associations between sensory and motor areas of the brain separated by several neuronal relays, within a few seconds. This paper proposes a model of fast learning along the dorsal pathway, from primary visual areas to pre-motor cortex. A new synaptic learning rule is proposed where synaptic efficacies converge rapidly toward a specific value determined by the number of active inputs of a neuron, respecting a principle of resource limitation in terms of total synaptic input efficacy available to a neuron. The efficacies are stable with regards to repeated arrival of spikes in a spike train. This rule reproduces the inverse relationship between initial and final synaptic efficacy observed in long-term potentiation (LTP) experiments. Simulations of learning experiments are conducted in a multilayer network of leaky integrate-and-fire (LIF) spiking neuron models. It is proposed that cortical feedback connections convey a top-down learning-enabling signal that guides bottom-up learning in “hidden” neurons that are not directly exposed to input or output activity. Simulations of repeated presentation of the same stimulus–response pair, show that, under conditions of fast learning with probabilistic synaptic transmission, the networks tend to recruit a new sub-network at each presentation to represent the association, rather than re-using a previously trained one. This increasing allocation of neural resources results in progressively shorter execution times, in line with experimentally observed reduction in response time with practice.This article is part of a Special Issue entitled: Neural Coding.

Input-rate modulation of gamma oscillations is sensitive to network topology, delays and short-term plasticity

Simulated networks of excitatory and inhibitory neurons have previously been shown to reproduce critical features of experimental data regarding neural coding in V1, such as a positive relationship between thalamic input spike rate and the power of gamma frequency oscillations. This effect, referred to as modulated gamma power, could represent a neural code in V1 for stimulus characteristics that affect thalamic spike rate such as contrast or intensity. The simulated network's assumptions included homogeneous random connectivity, equal synaptic delays after spike arrival, and constant synaptic efficacies. Plausible alternative assumptions include small world connectivity, a wide distribution of axonal propagation delays, and short term synaptic plasticity, and here we assess the individual impact of each of these on the model's success in reproducing modulated gamma power. First, we developed several alternative algorithms for simulating directed networks with clustered connectivity and balanced excitation and inhibition. We found that modulated gamma power was absent in all small-world networks that had a relatively low abundance of reciprocal connectivity, which suggests that such motifs are present in V1 cortical networks at levels at least equal to those found in random networks. We also found in a different network type that the balance of excitation and inhibition could be destroyed when the network was in the small-world regime. Given all neurons had identical in-degrees, this result suggests that balance relies on motif distributions as well as mean connectivity. Second, altering the distribution of axonal delays had little effect, but increasing the mean delay led to a secondary gamma modulation at harmonics of the main peak, and since this is not observed experimentally, it suggests a mean delay in V1 networks less than 2ms. Finally, we compared two types of excitatory synaptic plasticity, and found that modulated beta power emerged in addition to gamma power for one type, in the presence of short term depression in interneurons.This article is part of a Special Issue entitled “Neural Coding”.

Fail-safe detection of threshold crossings of linear integrate-and-fire neuron models in time-driven simulations

The characteristics of time-driven simulation are a fixed-size simulation step and a fixed-size communication interval [1]. The former defines update-and-check points, which are the discrete points in time when all neurons update their state variables and check for a super-threshold membrane potential. The latter defines the discrete points in time when all neurons communicate their spikes. The communication interval is a multiple of the simulation step size and limited only by the minimum synaptic transmission delay in the network. The time-driven environment of the simulator NEST [2] provides an 'on-grid' and an 'off-grid' framework that handle spikes differently. In the on-grid framework, spikes are incorporated, detected and emitted only at the pre-defined update-and-check points. In the off-grid framework, spikes can be incorporated and emitted at any point in time [3]. For each neuron the arrival times of incoming spikes introduce additional update-and-check points. Hence, the simulation step can be increased up to the size of the communication interval. The detection of a threshold crossing can only take place at a check point, but the timing of the referring spike is estimated with precision limited only by the limits of double representation. In general, a time-driven simulator that supports the off-grid framework performs neural network simulations with the same precision and faster than an event-driven simulator [4]. However, time-driven simulation still bears the risk of missing a threshold crossing as a very brief excursion of the membrane potential above threshold may not be detected at the next check point. In the off-grid framework, this problem is more pronounced in networks with low connectivity and strong coupling as well as in the case of low firing rates. The on-grid framework is even more affected due to fewer check points and the synchronized arrival of spikes. Here, we present algorithms which are guaranteed to detect all threshold crossings by supplementing the standard test for a super-threshold membrane potential at each check point and that exploit the information about the neuronal state at nearby check points. These additional tests need to be invoked whenever the membrane potential is sub-threshold, which means at virtually all check points. We develop sub-tests of increasing complexity and specificity, starting with simple sifting methods and ending up with a complex expression that faithfully indicates the existence of a threshold crossing between the last and the current check point. An analysis of the test specificities and computational costs results in a cascade of tests which locates all threshold crossings at a low computational cost.

Spatial filter by spike timing dependent plasticity

Spike timing dependent plasticity (STDP) is found in various areas of the brain, visual cortex, hippocampus and hindbrain of electric fish, etc. The synaptic modification by STDP depends on time difference between presynaptic spike arrival and postsynaptic firing. If presynaptic neuron fires earlier than postsynaptic neuron dose, synaptic weight is strengthened. If postsynaptic neuron fires earlier than presynaptic neuron dose, synaptic weight is weakened. This learning rule is one example of various rules (hippocampal type). The learning rule of electric fish type is reversed. Changes of synaptic efficiency precisely depend on timing of pre- and postsynaptic spikes under STDP. Because of this precise dependence, STDP is thought to play the important role in temporal processing of stimuli. Temporal processing by STDP is well known. However, the role of STDP in spatial processing is not enough understood. To investigate spatial processing by STDP, We studied the role in spatial processing on interconnected network with STDP, using computer simulation. In this study, we found two type spatial filter by STDP on interconnected network. One is low pass filter using the learning rule of electric fish type. Another is edge-enhancing filter using the learning rule of hippocampus type. In this study, we calculated the response of interconnected neural network with STDP learning. The structure of the network is one-dimensional array. The neuron of the network has connection to neighbour neurons. In the case of learning rule of electric fish type, the network provided the function of low pass filter. The network was stimulated by direct current rectangularly distributed with noise. We examined two cases. In one case, synaptic modification did not occur (Fig ​(Fig1a).1a). In another case, synaptic modification occurred (Fig ​(Fig1b).1b). In the first case, Noise presented in response of the network and covered the rectangular form of stimulus. In the second case, noise was eliminated and the rectangular form of stimulus was represented. In the case of learning rule of hippocampal type, the network provided the function of edge-enhancing filter. The network was stimulated by direct current rectangularly distributed. The firing rate of the neuron that represents the edge of stimulus was larger than one of the others. This result shows that the network with hipocampal type STDP plays the role of edge-enhancing filter. These results show that interconnected network with STDP provides the function of spatial filter, and suggest that STDP would plays the important role in spatial processing. Figure 1 Response of network. Vertical and horizontal line indicates response of the network and neuronal number, respectively. (a) and (b) show the response of the network without and with STDP learning, respectively.

Connectivity reflects coding: a model of voltage-based STDP with homeostasis

Electrophysiological connectivity patterns in cortex often have a few strong connections, which are sometimes bidirectional, among a lot of weak connections. To explain these connectivity patterns, we created a model of spike timing-dependent plasticity (STDP) in which synaptic changes depend on presynaptic spike arrival and the postsynaptic membrane potential, filtered with two different time constants. Our model describes several nonlinear effects that are observed in STDP experiments, as well as the voltage dependence of plasticity. We found that, in a simulated recurrent network of spiking neurons, our plasticity rule led not only to development of localized receptive fields but also to connectivity patterns that reflect the neural code. For temporal coding procedures with spatio-temporal input correlations, strong connections were predominantly unidirectional, whereas they were bidirectional under rate-coded input with spatial correlations only. Thus, variable connectivity patterns in the brain could reflect different coding principles across brain areas; moreover, our simulations suggested that plasticity is fast.

Spike-timing dependent plasticity

Spike Timing Dependent Plasticity (STDP) is a temporally asymmetric form of Hebbian learning induced by tight temporal correlations between the spikes of pre- and postsynaptic neurons. As with other forms of synaptic plasticity, it is widely believed that it underlies learning and information storage in the brain, as well as the development and refinement of neuronal circuits during brain development (e.g. Bi and Poo, 2001; Sjostrom et al., 2008). With STDP, repeated presynaptic spike arrival a few milliseconds before postsynaptic action potentials leads in many synapse types to long-term potentiation (LTP) of the synapses, whereas repeated spike arrival after postsynaptic spikes leads to long-term depression (LTD) of the same synapse. The change of the synapse plotted as a function of the relative timing of pre- and postsynaptic action potentials is called the STDP function or learning window and varies between synapse types. The rapid change of the STDP function with the relative timing of spikes suggests the possibility of temporal coding schemes on a millisecond time scale.

Voltage and spike timing interact in STDP - a unified model

A phenomenological model of synaptic plasticity is able to account for a large body of experimental data on spike-timing-dependent plasticity (STDP). The basic ingredient of the model is the correlation of presynaptic spike arrival with postsynaptic voltage. The local membrane voltage is used twice: a first term accounts for the instantaneous voltage and the second one for a low-pass filtered voltage trace. Spike-timing effects emerge as a special case. We hypothesize that the voltage dependence can explain differential effects of STDP in dendrites, since the amplitude and time course of backpropagating action potentials or dendritic spikes influences the plasticity results in the model. The dendritic effects are simulated by variable choices of voltage time course at the site of the synapse, i.e., without an explicit model of the spatial structure of the neuron.

Why is connectivity in barrel cortex different from that in visual cortex? - A plasticity model.

Event Abstract Back to Event Why is connectivity in barrel cortex different from that in visual cortex? - A plasticity model. Claudia Clopath1*, Lars Busing2, Eleni Vasilaki3 and Wulfram Gerstner3 1 LCN , Switzerland 2 Graz University for Technology , Austria 3 EPFL, LCN, Switzerland Electrophysiological connectivity patterns in cortex often show a few strong connections in a sea of weak connections. In some brain areas a large fraction of strong connections are bidirectional, in others they are mainly unidirectional. In order to explain these connectivity patterns, we use a model of Spike-Timing-Dependent Plasticity where synaptic changes depend on presynaptic spike arrival and the postsynaptic membrane potential, filtered with two different time constants. The model describes several nonlinear effects in STDP experiments, as well as the voltage dependence of plasticity under voltage clamp and classical paradigms of LTP/LTD induction. We show that in a simulated recurrent network of spiking neurons our plasticity rule leads not only to development of localized receptive fields, but also to connectivity patterns that reflect the neural code: for temporal coding paradigms with spatio-temporal input correlations, strong connections are predominantly unidirectional, whereas they are bidirectional under rate coded input with spatial correlations only. Thus variable connectivity patterns in the brain, mainly unidirectional in barrel cortex versus bidirectional in visual cortex, could reflect different coding principles across brain areas. Conference: Computational and Systems Neuroscience 2010, Salt Lake City, UT, United States, 25 Feb - 2 Mar, 2010. Presentation Type: Poster Presentation Topic: Poster session II Citation: Clopath C, Busing L, Vasilaki E and Gerstner W (2010). Why is connectivity in barrel cortex different from that in visual cortex? - A plasticity model.. Front. Neurosci. Conference Abstract: Computational and Systems Neuroscience 2010. doi: 10.3389/conf.fnins.2010.03.00220 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 04 Mar 2010; Published Online: 04 Mar 2010. * Correspondence: Claudia Clopath, LCN, Lausanne, Switzerland, c.clopath@imperial.ac.uk Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Claudia Clopath Lars Busing Eleni Vasilaki Wulfram Gerstner Google Claudia Clopath Lars Busing Eleni Vasilaki Wulfram Gerstner Google Scholar Claudia Clopath Lars Busing Eleni Vasilaki Wulfram Gerstner PubMed Claudia Clopath Lars Busing Eleni Vasilaki Wulfram Gerstner Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Spike-based reinforcement learning in continuous state and action space: when policy gradient methods fail.

Changes of synaptic connections between neurons are thought to be the physiological basis of learning. These changes can be gated by neuromodulators that encode the presence of reward. We study a family of reward-modulated synaptic learning rules for spiking neurons on a learning task in continuous space inspired by the Morris Water maze. The synaptic update rule modifies the release probability of synaptic transmission and depends on the timing of presynaptic spike arrival, postsynaptic action potentials, as well as the membrane potential of the postsynaptic neuron. The family of learning rules includes an optimal rule derived from policy gradient methods as well as reward modulated Hebbian learning. The synaptic update rule is implemented in a population of spiking neurons using a network architecture that combines feedforward input with lateral connections. Actions are represented by a population of hypothetical action cells with strong mexican-hat connectivity and are read out at theta frequency. We show that in this architecture, a standard policy gradient rule fails to solve the Morris watermaze task, whereas a variant with a Hebbian bias can learn the task within 20 trials, consistent with experiments. This result does not depend on implementation details such as the size of the neuronal populations. Our theoretical approach shows how learning new behaviors can be linked to reward-modulated plasticity at the level of single synapses and makes predictions about the voltage and spike-timing dependence of synaptic plasticity and the influence of neuromodulators such as dopamine. It is an important step towards connecting formal theories of reinforcement learning with neuronal and synaptic properties.

Connectivity reflects coding: A model of voltage-based spike-timing-dependent-plasticity with homeostasis

AbstractElectrophysiological connectivity patterns in cortex often show a few strong connections in a sea of weak connections. In some brain areas a large fraction of strong connections are bidirectional, in others they are mainly unidirectional. In order to explain these connectivity patterns, we use a model of Spike-Timing-Dependent Plasticity where synaptic changes depend on presynaptic spike arrival and the postsynaptic membrane potential. The model describes several nonlinear effects in STDP experiments, as well as the voltage dependence of plasticity under voltage clamp and classical paradigms of LTP/LTD induction. We show that in a simulated recurrent network of spiking neurons our plasticity rule leads not only to receptive field development, but also to connectivity patterns that reflect the neural code: for temporal coding paradigms strong connections are predominantly unidirectional, whereas they are bidirectional under rate coding. Thus variable connectivity patterns in the brain could reflect different coding principles across brain areas.

Estimating linear–nonlinear models using Rényi divergences

This article compares a family of methods for characterizing neural feature selectivity using natural stimuli in the framework of the linear–nonlinear model. In this model, the spike probability depends in a nonlinear way on a small number of stimulus dimensions. The relevant stimulus dimensions can be found by optimizing a Rényi divergence that quantifies a change in the stimulus distribution associated with the arrival of single spikes. Generally, good reconstructions can be obtained based on optimization of Rényi divergence of any order, even in the limit of small numbers of spikes. However, the smallest error is obtained when the Rényi divergence of order 1 is optimized. This type of optimization is equivalent to information maximization, and is shown to saturate the Cramér–Rao bound describing the smallest error allowed for any unbiased method. We also discuss conditions under which information maximization provides a convenient way to perform maximum likelihood estimation of linear–nonlinear models from neural data.